For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
kv_cache_row_offsets_ragged_paged
def kv_cache_row_offsets_ragged_paged[target: StringSpan[ImmStaticOrigin]](cache_row_offsets: TileTensor[.uint32, Engine=cache_row_offsets.Engine, linear_idx_type=cache_row_offsets.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type], cache_lengths: TileTensor[.uint32, Engine=cache_lengths.Engine, linear_idx_type=cache_lengths.linear_idx_type], ctx: DeviceContext)
Builds cumulative valid-cache row offsets for a ragged prefill batch.
Parameters:
- target (
StringSpan[ImmStaticOrigin]): Compilation target string; must be a GPU target.
Args:
- cache_row_offsets (
TileTensor[.uint32, Engine=cache_row_offsets.Engine, linear_idx_type=cache_row_offsets.linear_idx_type]): Output tensor of shape (batch_size + 1) receiving the cumulative valid-cache row offsets. - input_row_offsets (
TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type]): Tensor of shape (batch_size + 1) denoting the start of each sequence in the ragged batch. - cache_lengths (
TileTensor[.uint32, Engine=cache_lengths.Engine, linear_idx_type=cache_lengths.linear_idx_type]): Tensor of shape (batch_size,) holding the valid cache length for each batch. - ctx (
DeviceContext): The call context pointer, passed by the graph compiler.