IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

gated_delta_conv1d_fwd_gpu

def gated_delta_conv1d_fwd_gpu[work_dtype: DType, state_dtype: DType, KERNEL_SIZE: Int, CONV1D_BLOCK_DIM: Int, WRITE_STATE: Bool, qkv_input_ragged_LT: TensorLayout, conv_weight_LT: TensorLayout, conv_state_LT: TensorLayout, slot_idx_LT: TensorLayout, input_row_offsets_LT: TensorLayout, conv_output_ragged_LT: TensorLayout, Engine: TensorEngine, CAPTURE_INPUT: Bool = False, qkv_window_LT: TensorLayout = conv_output_ragged_LT, APPLY_SILU: Bool = False](batch_size: Int32, total_seq_len: Int32, conv_dim: Int32, tokens_per_block: Int32, window_row_len: Int32, num_draft_tokens: Int32, qkv_input_ragged: TileTensor[work_dtype, qkv_input_ragged_LT, ImmUntrackedOrigin, Engine=Engine], conv_weight: TileTensor[work_dtype, conv_weight_LT, ImmUntrackedOrigin, Engine=Engine], conv_state: TileTensor[state_dtype, conv_state_LT, MutUntrackedOrigin, Engine=Engine], slot_idx: TileTensor[.uint32, slot_idx_LT, ImmUntrackedOrigin, Engine=Engine], input_row_offsets: TileTensor[.uint32, input_row_offsets_LT, ImmUntrackedOrigin, Engine=Engine], conv_output_ragged: TileTensor[work_dtype, conv_output_ragged_LT, MutUntrackedOrigin, Engine=Engine], qkv_window: TileTensor[work_dtype, qkv_window_LT, MutUntrackedOrigin, Engine=Engine])

Slot-indexed causal depthwise conv1d over a ragged batch.

The thread owning a sequence's first token computes the outputs that see the old conv window and then rewrites the window; later tokens read only the ragged input. With WRITE_STATE False the window is left unchanged.

With CAPTURE_INPUT (a verify's forward launch) every thread of a row of at most window_row_len tokens also copies the input it owns to that row's slot of qkv_window, for the rollback to replay. A longer row is not copied: its head thread writes the window ending at the row's last real token, before the num_draft_tokens drafts every verify row ends with, so the rollback replays none of it. Both arguments are read only under CAPTURE_INPUT.

With APPLY_SILU every output is silu(conv_sum), the graph's SiLU on the fp32 sum, before the store rounds it to work_dtype.

Every thread waits for the preceding grid before it reads any operand, since each of them (the input, the weight, the offsets) can be that grid's output. The launch must carry pdl_launch_attributes; the grid releases its dependent after the last store.

Was this page helpful?