IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

short_conv_ring

Depthwise causal short convolution over a ring of past inputs.

A request's conv state is a ring indexed by absolute position: the input at position p lives at ring[p % R]. One launch computes x + conv(x) and commits the chunk:

  • Output: a tap at p - j reads x when it is inside the chunk, the ring when it is before the chunk, and zero before position zero.
  • Commit: each sequence's last R inputs are written into their entries.

Only the first width - 1 tokens of a chunk read the ring, and the commit overwrites entries they read. One thread therefore computes those tokens and then commits, so every read of an old entry precedes the write of its replacement in program order. Every other token reads only x.

One code path covers decode, prefill, mixed batches and speculative verify. A rejected draft token's entry is overwritten once the sequence passes its position again, so R must be width - 1 plus the largest rollback.

Functionsโ€‹

Was this page helpful?