IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mxfp4_index_k_cache

def mxfp4_index_k_cache[page_size: Int](cache: ImmTileTensor[.uint8, Engine=cache.Engine, address_space=cache.address_space, linear_idx_type=cache.linear_idx_type], lookup_table: ImmTileTensor[.uint32, Engine=lookup_table.Engine, address_space=lookup_table.address_space, linear_idx_type=lookup_table.linear_idx_type], cache_lengths: ImmTileTensor[.uint32, Engine=cache_lengths.Engine, address_space=cache_lengths.address_space, linear_idx_type=cache_lengths.linear_idx_type], max_seq_length: Int, max_cache_length: Int) -> MXFP4IndexKCache[page_size]

Views packed MXFP4 index-key pages as a paged cache.

Parameters:

  • ​page_size (Int): Keys per page.

Args:

Returns:

MXFP4IndexKCache[page_size]: The cache view.

def mxfp4_index_k_cache[page_size: Int](blocks: TileTensor[.uint8, Engine=blocks.Engine, address_space=blocks.address_space, linear_idx_type=blocks.linear_idx_type], page_stride: Int, layer: Int, lookup_table: TileTensor[.uint32, Engine=lookup_table.Engine, address_space=lookup_table.address_space, linear_idx_type=lookup_table.linear_idx_type], cache_lengths: TileTensor[.uint32, Engine=cache_lengths.Engine, address_space=cache_lengths.address_space, linear_idx_type=cache_lengths.linear_idx_type], max_seq_length: Int, max_cache_length: Int) -> MXFP4IndexKCache[page_size]

Views one layer of a multi-layer MXFP4 index-key leaf as a paged cache.

Parameters:

  • ​page_size (Int): Keys per page.

Args:

Returns:

MXFP4IndexKCache[page_size]: The cache view.

Raises:

If the leaf does not hold page_size MXFP4 records a page, its pitch or the layer is invalid, or the cache lengths or the lookup table are not row-major.

Was this page helpful?