For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
GatedDeltaConv1dFwd
struct GatedDeltaConv1dFwd[write_state: Bool = True, silu: Bool = False]
Gated DeltaNet causal conv1d forward pass (Pass 1 of two-pass prefill).
Reads/writes a single mutable conv-state pool of shape
[max_slots, conv_dim, kernel_size-1] in place at slot
slot_idx[batch_item]. No state-out tensor; the only output is the
per-token conv output. The pool's dtype is independent of the working
dtype, so the caller can keep the per-token tensors at fp32 while
storing the pool at the model's native dtype (bf16).
Tensor Shapes: - conv_output_ragged : [total_seq_len, conv_dim] (OUT) - qkv_input_ragged : [total_seq_len, conv_dim] - conv_weight : [conv_dim, kernel_size] - conv_state : [max_slots, conv_dim, kernel_size-1] (MUT) - slot_idx : [batch_size] uint32 - input_row_offsets : [batch_size + 1] uint32
Parameters
- write_state (
Bool): Whether to write the updated window back toconv_state. False only reads it for the look-back. - silu (
Bool): Whether each output issiluof the fp32 convolution sum, applied before the store rounds it to the output dtype.
Implemented traits
Methods
execute
static def execute(conv_output_ragged: T, qkv_input_ragged: T, conv_weight: T, conv_state: T, slot_idx: T, input_row_offsets: T, ctx: DeviceContext)
launch
static def launch[work_dtype: DType, state_dtype: DType, //, capture_input: Bool](conv_output_ragged: TileTensor[work_dtype, Engine=conv_output_ragged.Engine, address_space=conv_output_ragged.address_space, linear_idx_type=conv_output_ragged.linear_idx_type], qkv_input_ragged: TileTensor[work_dtype, Engine=qkv_input_ragged.Engine, address_space=qkv_input_ragged.address_space, linear_idx_type=qkv_input_ragged.linear_idx_type], conv_weight: TileTensor[work_dtype, Engine=conv_weight.Engine, address_space=conv_weight.address_space, linear_idx_type=conv_weight.linear_idx_type], conv_state: TileTensor[state_dtype, Engine=conv_state.Engine, address_space=conv_state.address_space, linear_idx_type=conv_state.linear_idx_type], slot_idx: TileTensor[.uint32, Engine=slot_idx.Engine, address_space=slot_idx.address_space, linear_idx_type=slot_idx.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, address_space=input_row_offsets.address_space, linear_idx_type=input_row_offsets.linear_idx_type], qkv_window: TileTensor[work_dtype, Engine=qkv_window.Engine, address_space=qkv_window.address_space, linear_idx_type=qkv_window.linear_idx_type], num_draft_tokens: Int, ctx: DeviceContext)
Launches the conv. With capture_input, a verify's forward launch at num_draft_tokens drafts, it copies each row that fits a slot of qkv_window to that slot and writes the window of every longer row at the row's last real token (see gated_delta_conv1d_fwd_gpu). qkv_window holds batch_size slots of equal length and is otherwise never touched.