For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
indexer_expand_candidates
def indexer_expand_candidates[candidates: Int, ratio: Int, block_kv: Int](input: TileTensor[.int32, Engine=input.Engine, address_space=input.address_space, linear_idx_type=input.linear_idx_type], starts: TileTensor[.int32, Engine=starts.Engine, address_space=starts.address_space, linear_idx_type=starts.linear_idx_type], stops: TileTensor[.int32, Engine=stops.Engine, address_space=stops.address_space, linear_idx_type=stops.linear_idx_type], output: TileTensor[.int32, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], ends: TileTensor[.int32, Engine=ends.Engine, address_space=ends.address_space, linear_idx_type=ends.linear_idx_type], ctx: DeviceContext)
Expand and sort candidate blocks; pad with the final valid block.
Each tensor is a row-major [rows, stride] tile whose row stride is
its second dimension. Candidates are nonnegative request-local block
ids or negative sentinels. Bounds are token positions in a shared
workspace; their base need not be block-aligned. Duplicates are
retained. ends counts valid token columns, including a partial final
block. All buffers are caller-owned and disjoint.
Parameters:
- candidates (
Int): Candidate capacity per row, a power of two. - ratio (
Int): Compression group size, either 1 or 2. - block_kv (
Int): Tokens per block, either 8 or 16.
Args:
- input (
TileTensor[.int32, Engine=input.Engine, address_space=input.address_space, linear_idx_type=input.linear_idx_type]): Candidate ids. Shape:[rows, input_stride]with at leastcandidatescolumns used. - starts (
TileTensor[.int32, Engine=starts.Engine, address_space=starts.address_space, linear_idx_type=starts.linear_idx_type]): First token position per row, in column 0. Shape:[rows, start_stride]. - stops (
TileTensor[.int32, Engine=stops.Engine, address_space=stops.address_space, linear_idx_type=stops.linear_idx_type]): Last-plus-one token position per row, in column 0. Shape:[rows, stop_stride]. - output (
TileTensor[.int32, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): Sorted expanded block ids. Shape:[rows, output_stride]with at leastcandidates * ratiocolumns used. - ends (
TileTensor[.int32, Engine=ends.Engine, address_space=ends.address_space, linear_idx_type=ends.linear_idx_type]): Valid token count per row, in column 0. Shape:[rows, end_stride]. - ctx (
DeviceContext): The device context.