For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mxfp4_index_k_cache
def mxfp4_index_k_cache[page_size: Int](cache: ImmTileTensor[.uint8, Engine=cache.Engine, address_space=cache.address_space, linear_idx_type=cache.linear_idx_type], lookup_table: ImmTileTensor[.uint32, Engine=lookup_table.Engine, address_space=lookup_table.address_space, linear_idx_type=lookup_table.linear_idx_type], cache_lengths: ImmTileTensor[.uint32, Engine=cache_lengths.Engine, address_space=cache_lengths.address_space, linear_idx_type=cache_lengths.linear_idx_type], max_seq_length: Int, max_cache_length: Int) -> MXFP4IndexKCache[page_size]
Views packed MXFP4 index-key pages as a paged cache.
Parameters:
- page_size (
Int): Keys per page.
Args:
- cache (
ImmTileTensor[.uint8, Engine=cache.Engine, address_space=cache.address_space, linear_idx_type=cache.linear_idx_type]): Pages written byindexer_k_storewithfp4=True,[pages, page_stride]row-major;page_strideis a multiple of 16 and holdspage_sizerecords ofMXFP4_INDEX_RECORD_BYTES. - lookup_table (
ImmTileTensor[.uint32, Engine=lookup_table.Engine, address_space=lookup_table.address_space, linear_idx_type=lookup_table.linear_idx_type]): Physical page of each request's logical page,[batch, max_pages]. - cache_lengths (
ImmTileTensor[.uint32, Engine=cache_lengths.Engine, address_space=cache_lengths.address_space, linear_idx_type=cache_lengths.linear_idx_type]): Tokens each request had cached before this call,[batch]. The keys this call scores are already stored. - max_seq_length (
Int): Most new tokens any request brings. - max_cache_length (
Int): Most cached tokens any request has.
Returns:
MXFP4IndexKCache[page_size]: The cache view.
def mxfp4_index_k_cache[page_size: Int](blocks: TileTensor[.uint8, Engine=blocks.Engine, address_space=blocks.address_space, linear_idx_type=blocks.linear_idx_type], page_stride: Int, layer: Int, lookup_table: TileTensor[.uint32, Engine=lookup_table.Engine, address_space=lookup_table.address_space, linear_idx_type=lookup_table.linear_idx_type], cache_lengths: TileTensor[.uint32, Engine=cache_lengths.Engine, address_space=cache_lengths.address_space, linear_idx_type=cache_lengths.linear_idx_type], max_seq_length: Int, max_cache_length: Int) -> MXFP4IndexKCache[page_size]
Views one layer of a multi-layer MXFP4 index-key leaf as a paged cache.
Parameters:
- page_size (
Int): Keys per page.
Args:
- blocks (
TileTensor[.uint8, Engine=blocks.Engine, address_space=blocks.address_space, linear_idx_type=blocks.linear_idx_type]): The leaf's blocks,[blocks, 1, layers, page_size, 1, record]withMXFP4_INDEX_RECORD_BYTESrecords, written byindexer_k_norm_rope_store_layerwithfp4=True. - page_stride (
Int): Bytes from one block to the next, or-1when packed. - layer (
Int): The layer's index within the leaf. - lookup_table (
TileTensor[.uint32, Engine=lookup_table.Engine, address_space=lookup_table.address_space, linear_idx_type=lookup_table.linear_idx_type]): Physical block of each request's logical page,[batch, max_pages]. - cache_lengths (
TileTensor[.uint32, Engine=cache_lengths.Engine, address_space=cache_lengths.address_space, linear_idx_type=cache_lengths.linear_idx_type]): Tokens each request had cached before this call,[batch]. The keys this call scores are already stored. - max_seq_length (
Int): Most new tokens any request brings. - max_cache_length (
Int): Most cached tokens any request has.
Returns:
MXFP4IndexKCache[page_size]: The cache view.
Raises:
If the leaf does not hold page_size MXFP4 records a page, its pitch
or the layer is invalid, or the cache lengths or the lookup table are
not row-major.