For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
read_mixed_split
def read_mixed_split(split_args: TileTensor[.uint32, Engine=split_args.Engine, address_space=split_args.address_space, linear_idx_type=split_args.linear_idx_type], batch: Int, total_q: Int, max_q_len: Int, max_cache_length: Int) -> Optional[MixedSplit]
Returns the split the host record describes, or None to keep the unsplit prefill route.
A record that is short, zero, or inconsistent with this batch's shape
keeps the unsplit route. The device offsets are not read here (that would
sync the host), so a record that passes these checks must come from the
same host values as input_row_offsets and the caches: if its head
tokens disagree with input_row_offsets, the views address the wrong
rows. Debug builds assert the agreement on the device.
One view holds the decode rows (max query length at most
MIXED_STEP_DECODE_WIDTH) and takes the decode routes; the other takes the
prefill route. Either view may lead, so the host keeps its own row order.
Args:
- split_args (
TileTensor[.uint32, Engine=split_args.Engine, address_space=split_args.address_space, linear_idx_type=split_args.linear_idx_type]): The host record of one replica batch,uint32[7](the KV cache inputmixed_step_split). It cuts the batch into a head view (rows[0, r)) and a tail view (rows[r, batch)): [0] head rowsr(0 = no split), [1] head tokens, [2] head max query length, [3] head max cache length, [4] tail max query length, [5] tail max cache length, [6] total context length of the prefill view. Slots 2-5 are the[1]max-length operands of the two cache views, so they keep the caches'uint32. - batch (
Int): Rows in the batch. - total_q (
Int): Query tokens in the batch. - max_q_len (
Int): The batch's max query length. - max_cache_length (
Int): The batch's max cache length.
Returns:
Optional[MixedSplit]: The validated split, or None.