For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

KVConnectorConfigInterface

KVConnectorConfigInterface

class max.nn.kv_cache.KVConnectorConfigInterface(*args, **kwargs)

source

Bases: Protocol

The KV connector configuration contract: a type plus per-tier settings.

Declared here because KVCacheParams carries it, and implemented by the Pydantic KVConnectorConfig in the pipelines layer (which owns CLI and config-file parsing). Structural typing keeps max.nn free of a Pydantic dependency, which the base max wheel does not ship, while still giving every consumer a checked type instead of Any.

block_store_endpoint

property block_store_endpoint: str | None

source

Endpoint for the co-located dKV service.

disk_offload_dir

property disk_offload_dir: str | None

source

Disk cache directory; None means auto-create one.

disk_offload_max_gb

property disk_offload_max_gb: float | None

source

Disk budget in GiB; None sizes it from the device pool.

host_offload_max_gb

property host_offload_max_gb: float | None

source

Host budget in GiB; None sizes it from the device pool.

host_staging_alloc

property host_staging_alloc: Literal['registered', 'driver']

source

Whether the host tier registers host memory or driver-allocates it.

host_staging_mlock

property host_staging_mlock: bool

source

Whether the registered host staging region is locked with mlock.

num_disk_workers

property num_disk_workers: int

source

Disk I/O worker threads for the tiered connector.

num_h2d_onload_lanes

property num_h2d_onload_lanes: int

source

Concurrent H2D onload lanes for the rust_tiered connector.

type

property type: KVConnectorType

source

Which off-device backing store to use.

use_disk_direct_io

property use_disk_direct_io: bool

source

Whether the disk tier’s file I/O bypasses the page cache.