For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
kv_cache_row_offsets_ragged_paged
def kv_cache_row_offsets_ragged_paged(cache_row_offsets: TileTensor[.uint32, Engine=cache_row_offsets.Engine, linear_idx_type=cache_row_offsets.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type], cache_lengths: TileTensor[.uint32, Engine=cache_lengths.Engine, linear_idx_type=cache_lengths.linear_idx_type], ctx: DeviceContext)
Builds cumulative valid-cache row offsets for a ragged prefill batch.
Args:
- βcache_row_offsets (
TileTensor[.uint32, Engine=cache_row_offsets.Engine, linear_idx_type=cache_row_offsets.linear_idx_type]): Output tensor of shape (batch_size + 1) receiving the cumulative valid-cache row offsets. - βinput_row_offsets (
TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type]): Tensor of shape (batch_size + 1) denoting the start of each sequence in the ragged batch. - βcache_lengths (
TileTensor[.uint32, Engine=cache_lengths.Engine, linear_idx_type=cache_lengths.linear_idx_type]): Tensor of shape (batch_size,) holding the valid cache length for each batch. - βctx (
DeviceContext): The call context pointer, passed by the graph compiler.