For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
short_conv_ring
Depthwise causal short convolution over a ring of past inputs.
A request's conv state is a ring indexed by absolute position: the input at
position p lives at ring[p % R]. One launch computes x + conv(x) and
commits the chunk:
- Output: a tap at
p - jreadsxwhen it is inside the chunk, the ring when it is before the chunk, and zero before position zero. - Commit: each sequence's last
Rinputs are written into their entries.
Only the first width - 1 tokens of a chunk read the ring, and the commit
overwrites entries they read. One thread therefore computes those tokens
and then commits, so every read of an old entry precedes the write of its
replacement in program order. Every other token reads only x.
One code path covers decode, prefill, mixed batches and speculative verify.
A rejected draft token's entry is overwritten once the sequence passes its
position again, so R must be width - 1 plus the largest rollback.
Functionsโ
- โ
fused_qk_rms_norm_short_conv_ragged_paged: A short-conv attention block's prologue in one GPU launch. - โ
short_conv_ring:x + conv(x)over a ragged batch, committing each sequence's lastRinputs into the ring whencommitis set.