For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ScratchKVLeafRegion

ScratchKVLeafRegion

class max.nn.kv_cache.ScratchKVLeafRegion(leaf_id, group_id, bytes_per_page, region, *, stride_alignment=None)

source

Bases: RecurrentKVLeafRegion

A row-addressed leaf drawn once per request and never published.

Addressed exactly like RecurrentKVLeafRegion, one row per layer, but its block does not rotate at a page boundary and carries no hash, so a request holds the same one from its first forward to its release.

Parameters:

• leaf_id (str)
• group_id (KVCacheGroupId)
• bytes_per_page (int)
• region (RecurrentStateRegion)
• stride_alignment (int | None)

blocks_to_reserve()

blocks_to_reserve(num_blocks, *, enable_prefix_caching)

source

Returns one: the block the request holds for its whole life.

Parameters:

• num_blocks (int)
• enable_prefix_caching (bool)

Return type:

int