For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

KVConnectorType

KVConnectorType

class max.nn.kv_cache.KVConnectorType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

source

Bases: str, Enum

Identifies which off-device backing store the KV cache uses.

Set on the connector config’s type field to control whether evicted cache pages stay on device only, tier across host and disk, or route through a distributed block store.

dkv

dkv = 'dkv'

source

Routes pages through a distributed KV block store.

Requires a block_store_endpoint on the connector config.

null

null = 'null'

source

No off-device backing store. Pages live on device only.

rust_tiered

rust_tiered = 'rust_tiered'

source

Tiers evicted pages across host memory and disk, backed by the Rust kv_tier_connector extension.

The only host/disk tiered implementation, and what tiered now resolves to: it runs its copies and disk I/O on Rust threads (no GIL contention) and overlaps onloads with GPU compute via asynchronous transfer handles. Requires enable_prefix_caching, host_offload_max_gb, and a disk_offload_dir on the connector config. Raises on non-CUDA/HIP devices.

tiered

tiered = 'tiered'

source

Tiers evicted pages across host memory and disk.

Requires enable_prefix_caching, host_offload_max_gb, and a disk_offload_dir on the connector config.

Deprecated

Deprecated since version A: backward-compatible alias for rust_tiered; the Python implementation was removed.