For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

PagedKVLeafRegion

PagedKVLeafRegion

class max.nn.kv_cache.PagedKVLeafRegion(leaf_id, group_id, bytes_per_page, page_size)

source

Bases: KVLeafRegion

A leaf the graph reaches through a per-forward page table.

Parameters:

• leaf_id (str)
• group_id (KVCacheGroupId)
• bytes_per_page (int)
• page_size (int)

blocks_to_reserve()

blocks_to_reserve(num_blocks)

source

Returns num_blocks, capped by the window when the leaf has one.

Parameters:

num_blocks (int)

Return type:

int

page_size

page_size: int

source

Number of tokens per page.