For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

short_conv_ring

Depthwise causal short convolution over a ring of past inputs.

A request's conv state is a ring indexed by absolute position: the input at position p lives at ring[p % R]. A forward is two launches that never touch the same entry:

• short_conv_ring_fwd: x + conv(x). A tap at p - j reads x when it is inside the chunk, the ring when it is before the chunk, and zero before position zero. Writes nothing.
• short_conv_ring_commit: writes each sequence's last R inputs into their entries. Reads nothing.

One code path therefore covers decode, prefill, mixed batches and speculative verify. A rejected draft token's entry is overwritten once the sequence passes its position again, so R must be width - 1 plus the largest rollback.

Functions

• ​fused_qk_rms_norm_short_conv_ragged_paged: A short-conv attention block's prologue in one GPU launch.
• ​short_conv_ring_commit: Writes each sequence's last R rows of x into its ring slot.
• ​short_conv_ring_commit_kv: Commits the K and V conv inputs of an attention block in one launch.
• ​short_conv_ring_fwd: x + conv(x) over a ragged batch; reads the ring, writes nothing.