For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
PagedKVLeafRegion
PagedKVLeafRegion
class max.nn.kv_cache.PagedKVLeafRegion(leaf_id, group_id, bytes_per_page, page_size, *, row_bytes=1)
Bases: KVLeafRegion
A leaf the graph reaches through a per-forward page table.
-
Parameters:
-
- leaf_id (str)
- group_id (KVCacheGroupId)
- bytes_per_page (int)
- page_size (int)
- row_bytes (int)
blocks_to_reserve()
blocks_to_reserve(num_blocks, *, enable_prefix_caching)
Returns num_blocks, capped by the window when the leaf has one.
A sliding-window leaf only needs the pages holding the tokens its window looks back at, plus the page it is writing now.
page_size
page_size: int
Number of tokens per page.
staged_input_shapes()
staged_input_shapes(batch_size, num_blocks)
Returns this leaf’s lookup table.
write_staged_inputs()
write_staged_inputs(plans, into)
Writes each row’s block ids, zeroing the slots it does not reach.