For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
gated_delta_conv1d_fwd_gpu
def gated_delta_conv1d_fwd_gpu[work_dtype: DType, state_dtype: DType, KERNEL_SIZE: Int, CONV1D_BLOCK_DIM: Int, WRITE_STATE: Bool, qkv_input_ragged_LT: TensorLayout, conv_weight_LT: TensorLayout, conv_state_LT: TensorLayout, slot_idx_LT: TensorLayout, input_row_offsets_LT: TensorLayout, conv_output_ragged_LT: TensorLayout, Engine: TensorEngine, CAPTURE_INPUT: Bool = False, qkv_window_LT: TensorLayout = conv_output_ragged_LT, APPLY_SILU: Bool = False](batch_size: Int32, total_seq_len: Int32, conv_dim: Int32, tokens_per_block: Int32, window_row_len: Int32, num_draft_tokens: Int32, qkv_input_ragged: TileTensor[work_dtype, qkv_input_ragged_LT, ImmUntrackedOrigin, Engine=Engine], conv_weight: TileTensor[work_dtype, conv_weight_LT, ImmUntrackedOrigin, Engine=Engine], conv_state: TileTensor[state_dtype, conv_state_LT, MutUntrackedOrigin, Engine=Engine], slot_idx: TileTensor[.uint32, slot_idx_LT, ImmUntrackedOrigin, Engine=Engine], input_row_offsets: TileTensor[.uint32, input_row_offsets_LT, ImmUntrackedOrigin, Engine=Engine], conv_output_ragged: TileTensor[work_dtype, conv_output_ragged_LT, MutUntrackedOrigin, Engine=Engine], qkv_window: TileTensor[work_dtype, qkv_window_LT, MutUntrackedOrigin, Engine=Engine])
Slot-indexed causal depthwise conv1d over a ragged batch.
The thread owning a sequence's first token computes the outputs that
see the old conv window and then rewrites the window; later tokens read
only the ragged input. With WRITE_STATE False the window is left
unchanged.
With CAPTURE_INPUT (a verify's forward launch) every thread of a row
of at most window_row_len tokens also copies the input it owns to that
row's slot of qkv_window, for the rollback to replay. A longer row is
not copied: its head thread writes the window ending at the row's last
real token, before the num_draft_tokens drafts every verify row ends
with, so the rollback replays none of it. Both arguments are read only
under CAPTURE_INPUT.
With APPLY_SILU every output is silu(conv_sum), the graph's SiLU on
the fp32 sum, before the store rounds it to work_dtype.
Every thread waits for the preceding grid before it reads any operand,
since each of them (the input, the weight, the offsets) can be that
grid's output. The launch must carry pdl_launch_attributes; the grid
releases its dependent after the last store.