IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

GatedDeltaConv1dFwd

struct GatedDeltaConv1dFwd[write_state: Bool = True, silu: Bool = False]

Gated DeltaNet causal conv1d forward pass (Pass 1 of two-pass prefill).

Reads/writes a single mutable conv-state pool of shape [max_slots, conv_dim, kernel_size-1] in place at slot slot_idx[batch_item]. No state-out tensor; the only output is the per-token conv output. The pool's dtype is independent of the working dtype, so the caller can keep the per-token tensors at fp32 while storing the pool at the model's native dtype (bf16).

Tensor Shapes: - conv_output_ragged : [total_seq_len, conv_dim] (OUT) - qkv_input_ragged : [total_seq_len, conv_dim] - conv_weight : [conv_dim, kernel_size] - conv_state : [max_slots, conv_dim, kernel_size-1] (MUT) - slot_idx : [batch_size] uint32 - input_row_offsets : [batch_size + 1] uint32

Parameters​

  • ​write_state (Bool): Whether to write the updated window back to conv_state. False only reads it for the look-back.
  • ​silu (Bool): Whether each output is silu of the fp32 convolution sum, applied before the store rounds it to the output dtype.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute(conv_output_ragged: T, qkv_input_ragged: T, conv_weight: T, conv_state: T, slot_idx: T, input_row_offsets: T, ctx: DeviceContext)

launch​

static def launch[work_dtype: DType, state_dtype: DType, //, capture_input: Bool](conv_output_ragged: TileTensor[work_dtype, Engine=conv_output_ragged.Engine, address_space=conv_output_ragged.address_space, linear_idx_type=conv_output_ragged.linear_idx_type], qkv_input_ragged: TileTensor[work_dtype, Engine=qkv_input_ragged.Engine, address_space=qkv_input_ragged.address_space, linear_idx_type=qkv_input_ragged.linear_idx_type], conv_weight: TileTensor[work_dtype, Engine=conv_weight.Engine, address_space=conv_weight.address_space, linear_idx_type=conv_weight.linear_idx_type], conv_state: TileTensor[state_dtype, Engine=conv_state.Engine, address_space=conv_state.address_space, linear_idx_type=conv_state.linear_idx_type], slot_idx: TileTensor[.uint32, Engine=slot_idx.Engine, address_space=slot_idx.address_space, linear_idx_type=slot_idx.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, address_space=input_row_offsets.address_space, linear_idx_type=input_row_offsets.linear_idx_type], qkv_window: TileTensor[work_dtype, Engine=qkv_window.Engine, address_space=qkv_window.address_space, linear_idx_type=qkv_window.linear_idx_type], num_draft_tokens: Int, ctx: DeviceContext)

Launches the conv. With capture_input, a verify's forward launch at num_draft_tokens drafts, it copies each row that fits a slot of qkv_window to that slot and writes the window of every longer row at the row's last real token (see gated_delta_conv1d_fwd_gpu). qkv_window holds batch_size slots of equal length and is otherwise never touched.

Was this page helpful?