For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
indexer_score_ragged_paged
def indexer_score_ragged_paged[cache_t: KVCacheT, q_type: DType, //, num_heads: Int](output: TileTensor[.float32, Engine=output.Engine, linear_idx_type=output.linear_idx_type], q: TileTensor[q_type, Engine=q.Engine, linear_idx_type=q.linear_idx_type], weights: TileTensor[.float32, Engine=weights.Engine, linear_idx_type=weights.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type], base: TileTensor[.int32, Engine=base.Engine, linear_idx_type=base.linear_idx_type], cutoff: TileTensor[.int32, Engine=cutoff.Engine, linear_idx_type=cutoff.linear_idx_type], cache: cache_t, ctx: DeviceContext)
Scores every query row against its live compressed entries.
Parameters:
- cache_t (
KVCacheT): The compressed leaf's key cache type (inferred); one head, paged by entry throughslots_per_page. - q_type (
DType): Query element type (inferred). - num_heads (
Int): Query heads summed into each score.
Args:
- output (
TileTensor[.float32, Engine=output.Engine, linear_idx_type=output.linear_idx_type]):[num_rows, num_cand]scores,num_candeven; columnsnum_cand // 2on are the fresh windows. - q (
TileTensor[q_type, Engine=q.Engine, linear_idx_type=q.linear_idx_type]):[num_rows, num_heads, head_dim], the last axis contiguous. - weights (
TileTensor[.float32, Engine=weights.Engine, linear_idx_type=weights.linear_idx_type]):[num_rows, num_heads]per-head weights, contiguous. - input_row_offsets (
TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type]):[batch + 1]ragged row offsets. - base (
TileTensor[.int32, Engine=base.Engine, linear_idx_type=base.linear_idx_type]):[batch]entries each request had closed before this chunk. - cutoff (
TileTensor[.int32, Engine=cutoff.Engine, linear_idx_type=cutoff.linear_idx_type]):[num_rows]entries each query may see, counting from 0. - cache (
cache_t): This layer's compressed leaf. - ctx (
DeviceContext): Device context used to enqueue the GPU kernel.