For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

KVConnectorType

KVConnectorType

class max.nn.kv_cache.KVConnectorType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

source

Bases: str, Enum

Identifies which off-device backing store the KV cache uses.

Set on the connector config’s type field to control whether evicted cache pages stay on device only, tier across host and disk, or route through a distributed block store. MAX currently supports only null.

dkv

dkv = 'dkv'

source

Routes pages through a distributed KV block store.

MAX currently doesn’t support this connector type.

Requires a block_store_endpoint on the connector config.

null

null = 'null'

source

No off-device backing store. Pages live on device only.

rust_tiered

rust_tiered = 'rust_tiered'

source

Tiers evicted pages across host memory and disk, backed by the Rust kv_tier_connector extension.

MAX currently doesn’t support this connector type.

The only host/disk tiered implementation, and what tiered now resolves to: it runs its copies and disk I/O on Rust threads (no GIL contention) and overlaps onloads with GPU compute via asynchronous transfer handles. Requires enable_prefix_caching. Raises on non-CUDA/HIP devices.

tiered

tiered = 'tiered'

source

Tiers evicted pages across host memory and disk.

MAX currently doesn’t support this connector type.

Requires enable_prefix_caching.

Deprecated

Deprecated since version A: backward-compatible alias for rust_tiered; the Python implementation was removed.