For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

KVTransferEngineMetadata

KVTransferEngineMetadata

class max.pipelines.kv_cache.KVTransferEngineMetadata(leaf_ids, num_pages_per_group, bytes_per_page, bytes_per_group, replicated_per_group=<factory>, *, name, memory_type, hostname, agents_meta)

source

Bases: TransferEngineMetadata

Metadata associated with a KV cache transfer engine.

Extends the transport-only TransferEngineMetadata with the KV-cache/topology fields (page geometry and TP replication).

This is safe to send between threads/processes.

Parameters:

• leaf_ids (list[str])
• num_pages_per_group (list[int])
• bytes_per_page (int)
• bytes_per_group (list[int])
• replicated_per_group (list[bool])
• name (str)
• memory_type (MemoryType)
• hostname (str)
• agents_meta (list[list[TensorAgentMetadata]])

bytes_per_group

bytes_per_group: list[int]

source

Bytes per page for each tensor group, one entry per NIXL group. The first entry is the main group; subsequent entries correspond to extra groups, such as draft KV in speculative decoding. bytes_per_page equals sum(bytes_per_group).

bytes_per_page

bytes_per_page: int

source

Bytes per page for each tensor.

leaf_ids

leaf_ids: list[str]

source

Id of each NIXL group, parallel to bytes_per_group. Transfers address pages by these ids. KVTransferEngine.from_paged_kv_cache() uses the cache’s leaf ids.

num_pages_per_group

num_pages_per_group: list[int]

source

Total number of pages in each group, including the null block. Parallel to bytes_per_group.

replicated_per_group

replicated_per_group: list[bool]

source

Per-group TP replication, parallel to bytes_per_group. True entries are replicated identically across TP shards (MLA-style); False entries are sharded. This is the sole replication datum on the wire; the peer’s resolve_transfer_strategy() plans each group from it.