IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

GatedDeltaConv1dVerifyFwd

struct GatedDeltaConv1dVerifyFwd[rollback: Bool, silu: Bool = False]

gated_delta_conv1d_fwd in a speculative verify graph.

The verify width K is the length of verify_width, whose contents are never read. At K == 0 the window has no draft to reject, so the forward launch writes it and the rollback launch does nothing. At K > 0 the forward launch only reads it and the rollback launch places it at the accepted length.

A rollback launch at K == 0 leaves conv_output_ragged unwritten.

The forward launch at K > 0 also copies each row that fits a slot of qkv_window, as one token and its K drafts do, into that slot for the rollback to replay. A longer row, as a prompt chunk mixed into the batch, is not copied: the forward writes its window at its last real token, so the copy stays batch_size slots long whatever the chunk. Every other launch leaves qkv_window unwritten.

Tensor Shapes: As gated_delta_conv1d_fwd, plus - qkv_window : [batch_size * slot_len, conv_dim] (OUT) - verify_width : [K] int64

Parameters​

  • ​rollback (Bool): Whether this is the rollback's launch.
  • ​silu (Bool): Whether the forward launch stores its output through SiLU, as gated_delta_conv1d_fwd's silu does. The rollback launch's output is never read, so it ignores this.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute(conv_output_ragged: T, qkv_window: T, qkv_input_ragged: T, conv_weight: T, conv_state: T, slot_idx: T, input_row_offsets: T, verify_width: T, ctx: DeviceContext)

Was this page helpful?