For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
KVTransferEngineMetadata
KVTransferEngineMetadata
class max.pipelines.kv_cache.KVTransferEngineMetadata(total_num_pages, bytes_per_page, bytes_per_group, replicated_per_group=<factory>, *, name, memory_type, hostname, agents_meta)
Bases: TransferEngineMetadata
Metadata associated with a KV cache transfer engine.
Extends the transport-only TransferEngineMetadata with the
KV-cache/topology fields (page geometry and TP replication).
This is safe to send between threads/processes.
-
Parameters:
bytes_per_group
Bytes per page for each tensor group, one entry per NIXL group. The
first entry is the main group; subsequent entries correspond to extra
groups (e.g., draft KV in speculative decoding). bytes_per_page
equals sum(bytes_per_group).
bytes_per_page
bytes_per_page: int
Bytes per page for each tensor.
replicated_per_group
Per-group TP replication, parallel to bytes_per_group. True
entries are replicated identically across TP shards (MLA-style); False
entries are sharded. This is the sole replication datum on the wire; the
peer’s resolve_transfer_strategy() plans each group from it.
total_num_pages
total_num_pages: int
Total number of pages in each tensor.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!