For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
GatedDeltaConv1dVerifyFwd
struct GatedDeltaConv1dVerifyFwd[rollback: Bool, silu: Bool = False]
gated_delta_conv1d_fwd in a speculative verify graph.
The verify width K is the length of verify_width, whose contents are
never read. At K == 0 the window has no draft to reject, so the forward
launch writes it and the rollback launch does nothing. At K > 0 the
forward launch only reads it and the rollback launch places it at the
accepted length.
A rollback launch at K == 0 leaves conv_output_ragged unwritten.
The forward launch at K > 0 also copies each row that fits a slot of
qkv_window, as one token and its K drafts do, into that slot for the
rollback to replay. A longer row, as a prompt chunk mixed into the batch,
is not copied: the forward writes its window at its last real token, so
the copy stays batch_size slots long whatever the chunk. Every other
launch leaves qkv_window unwritten.
Tensor Shapes:
As gated_delta_conv1d_fwd, plus
- qkv_window : [batch_size * slot_len, conv_dim] (OUT)
- verify_width : [K] int64
Parameters
- rollback (
Bool): Whether this is the rollback's launch. - silu (
Bool): Whether the forward launch stores its output through SiLU, asgated_delta_conv1d_fwd'ssiludoes. The rollback launch's output is never read, so it ignores this.
Implemented traits
Methods
execute
static def execute(conv_output_ragged: T, qkv_window: T, qkv_input_ragged: T, conv_weight: T, conv_state: T, slot_idx: T, input_row_offsets: T, verify_width: T, ctx: DeviceContext)