For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

kv_cache_row_offsets_ragged_paged

def kv_cache_row_offsets_ragged_paged[target: StringSlice[ImmStaticOrigin]](cache_row_offsets: TileTensor[DType.uint32, Storage=cache_row_offsets.Storage, linear_idx_type=cache_row_offsets.linear_idx_type], input_row_offsets: TileTensor[DType.uint32, Storage=input_row_offsets.Storage, linear_idx_type=input_row_offsets.linear_idx_type], cache_lengths: TileTensor[DType.uint32, Storage=cache_lengths.Storage, linear_idx_type=cache_lengths.linear_idx_type], ctx: DeviceContext)

Builds cumulative valid-cache row offsets for a ragged prefill batch.

Parameters:

target (StringSlice[ImmStaticOrigin]): Compilation target string; must be a GPU target.

Args:

cache_row_offsets (TileTensor[DType.uint32, Storage=cache_row_offsets.Storage, linear_idx_type=cache_row_offsets.linear_idx_type]): Output tensor of shape (batch_size + 1) receiving the cumulative valid-cache row offsets.
input_row_offsets (TileTensor[DType.uint32, Storage=input_row_offsets.Storage, linear_idx_type=input_row_offsets.linear_idx_type]): Tensor of shape (batch_size + 1) denoting the start of each sequence in the ragged batch.
cache_lengths (TileTensor[DType.uint32, Storage=cache_lengths.Storage, linear_idx_type=cache_lengths.linear_idx_type]): Tensor of shape (batch_size,) holding the valid cache length for each batch.
ctx (DeviceContext): The call context pointer, passed by the graph compiler.