For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
KVCacheMemory
KVCacheMemory
class max.nn.kv_cache.KVCacheMemory(replicated, buffers)
Bases: object
One logical (child, kind) KV tensor as per-TP-shard uint8 views.
A unit is one logical tensor — a cache’s values or its scales —
holding a 2-D [num_pages, bytes_per_page] view per TP shard in canonical
device order.
replicated indicates that all buffers hold identical bytes. This is true
for certain cases like TP + MLA, TP + MiniMaxM3IndexerAttn, etc.
buffers
bytes_per_page
property bytes_per_page: int
Returns the per-page byte stride shared by every shard.
host_bytes_per_page
property host_bytes_per_page: int
Returns the width of one host block row holding this unit’s page.
A replicated (MLA) unit contributes its stride once – one copy is stored and broadcast back on load, so counting its peers would double the pinned host allocation. Must match across replicas, so a block written by one is readable by another.
replicated
replicated: bool
total_num_pages
property total_num_pages: int
Returns the total number of pages (including the null block).