IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

read_mixed_split

def read_mixed_split(split_args: TileTensor[.uint32, Engine=split_args.Engine, address_space=split_args.address_space, linear_idx_type=split_args.linear_idx_type], batch: Int, total_q: Int, max_q_len: Int, max_cache_length: Int) -> Optional[MixedSplit]

Returns the split the host record describes, or None to keep the unsplit prefill route.

A record that is short, zero, or inconsistent with this batch's shape keeps the unsplit route. The device offsets are not read here (that would sync the host), so a record that passes these checks must come from the same host values as input_row_offsets and the caches: if its head tokens disagree with input_row_offsets, the views address the wrong rows. Debug builds assert the agreement on the device.

One view holds the decode rows (max query length at most MIXED_STEP_DECODE_WIDTH) and takes the decode routes; the other takes the prefill route. Either view may lead, so the host keeps its own row order.

Args:

  • ​split_args (TileTensor[.uint32, Engine=split_args.Engine, address_space=split_args.address_space, linear_idx_type=split_args.linear_idx_type]): The host record of one replica batch, uint32[7] (the KV cache input mixed_step_split). It cuts the batch into a head view (rows [0, r)) and a tail view (rows [r, batch)): [0] head rows r (0 = no split), [1] head tokens, [2] head max query length, [3] head max cache length, [4] tail max query length, [5] tail max cache length, [6] total context length of the prefill view. Slots 2-5 are the [1] max-length operands of the two cache views, so they keep the caches' uint32.
  • ​batch (Int): Rows in the batch.
  • ​total_q (Int): Query tokens in the batch.
  • ​max_q_len (Int): The batch's max query length.
  • ​max_cache_length (Int): The batch's max cache length.

Returns:

Optional[MixedSplit]: The validated split, or None.

Was this page helpful?