For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

KVConnectorConfig

KVConnectorConfig

class max.pipelines.kv_cache.KVConnectorConfig(*, config_file=None, section_name=None, host_kvcache_swap_space_gb=50.0, disk_offload_dir=None, disk_offload_max_gb=50.0, num_disk_workers=32, block_store_endpoint=None, **extra_data)

source

Bases: ConfigFileModel

Connector-specific configuration for KV cache connectors.

Common fields are typed. Additional connector-specific fields pass through via extra="allow" and are accessible via model_extra.

Parameters:

• config_file (str | None)
• section_name (str | None)
• host_kvcache_swap_space_gb (float)
• disk_offload_dir (str | None)
• disk_offload_max_gb (float)
• num_disk_workers (int)
• block_store_endpoint (str | None)
• extra_data (Any)

block_store_endpoint

block_store_endpoint: str | None

source

Endpoint for the co-located dKV service.

Remote dKV endpoints are discovered at runtime through the Orchestrator (via external_block_metadata on the request context), not configured statically. For multi-store reads, the discovered metadata must include MAX-native transfer-engine metadata so the connector can reuse KVTransferEngine.connect().

disk_offload_dir

disk_offload_dir: str | None

source

Directory for disk-based KV cache offloading.

disk_offload_max_gb

disk_offload_max_gb: float

source

Maximum disk space in GB for KV cache offloading.

host_kvcache_swap_space_gb

host_kvcache_swap_space_gb: float

source

Host memory in GiB for KV cache swapping.

model_config

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

num_disk_workers

num_disk_workers: int

source

Number of disk I/O worker threads for the tiered connectors.