For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

KVTransferEngineMetadata

KVTransferEngineMetadata

class max.pipelines.kv_cache.KVTransferEngineMetadata(total_num_pages, bytes_per_page, bytes_per_group, replicated_per_group=<factory>, *, name, memory_type, hostname, agents_meta)

source

Bases: TransferEngineMetadata

Metadata associated with a KV cache transfer engine.

Extends the transport-only TransferEngineMetadata with the KV-cache/topology fields (page geometry and TP replication).

This is safe to send between threads/processes.

Parameters:

• total_num_pages (int)
• bytes_per_page (int)
• bytes_per_group (list[int])
• replicated_per_group (list[bool])
• name (str)
• memory_type (MemoryType)
• hostname (str)
• agents_meta (list[list[TensorAgentMetadata]])

bytes_per_group

bytes_per_group: list[int]

source

Bytes per page for each tensor group, one entry per NIXL group. The first entry is the main group; subsequent entries correspond to extra groups (e.g., draft KV in speculative decoding). bytes_per_page equals sum(bytes_per_group).

bytes_per_page

bytes_per_page: int

source

Bytes per page for each tensor.

replicated_per_group

replicated_per_group: list[bool]

source

Per-group TP replication, parallel to bytes_per_group. True entries are replicated identically across TP shards (MLA-style); False entries are sharded. This is the sole replication datum on the wire; the peer’s resolve_transfer_strategy() plans each group from it.

total_num_pages

total_num_pages: int

source

Total number of pages in each tensor.