For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
mixed_step_split
The mixed-step split record of a paged KV cache input.
A mixed step batches decode rows with prefill rows, so an op that picks one
route from its longest row sends the decode rows down the prefill route. The
KV cache managers fill a host record (mixed_step_split,
uint32[MIXED_STEP_SPLIT_SIZE]) for every batch, which cuts one replica batch
into a head and a tail view; an op validates it with read_mixed_split, runs
each view through its own route, and keeps the unsplit route for any record or
operand it cannot use.
comptime values
MIXED_STEP_DECODE_WIDTH
comptime MIXED_STEP_DECODE_WIDTH = 8
MIXED_STEP_SPLIT_SIZE
comptime MIXED_STEP_SPLIT_SIZE = 7
SPLIT_HEAD_LENGTHS_SLOT
comptime SPLIT_HEAD_LENGTHS_SLOT = 2
SPLIT_PREFILL_CONTEXT_SLOT
comptime SPLIT_PREFILL_CONTEXT_SLOT = 6
SPLIT_TAIL_LENGTHS_SLOT
comptime SPLIT_TAIL_LENGTHS_SLOT = 4
Structs
-
MixedSplit: A validated split of one mixed batch into a head and a tail view. -
SplitView: One row range of a split batch and its max lengths.
Functions
-
is_packed: Whethertis packed row-major. -
read_mixed_split: Returns the split the host record describes, or None to keep the unsplit prefill route. -
rebase_row_offsets: Returns the tail view's offsets rebased to start at 0, in one 1-CTA launch:[0, view.rows]query offsets, then the cache offsets whenwith_cache_offsets.