For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

NullKVConnectorConfig

NullKVConnectorConfig

class max.nn.kv_cache.NullKVConnectorConfig(type=KVConnectorType.null, host_offload_max_gb=None, host_staging_alloc='registered', host_staging_mlock=False, disk_offload_max_gb=None, disk_offload_dir=None, num_disk_workers=32, num_h2d_onload_lanes=1, use_disk_direct_io=False, block_store_endpoint=None)

source

Bases: object

Connector config for no off-device backing store.

The default for KVCacheParams.kv_connector_config, so the field is never None and every reader can go straight to .type.

Parameters:

• type (KVConnectorType)
• host_offload_max_gb (float | None)
• host_staging_alloc (Literal['registered', 'driver'])
• host_staging_mlock (bool)
• disk_offload_max_gb (float | None)
• disk_offload_dir (str | None)
• num_disk_workers (int)
• num_h2d_onload_lanes (int)
• use_disk_direct_io (bool)
• block_store_endpoint (str | None)

block_store_endpoint

block_store_endpoint: str | None = None

source

disk_offload_dir

disk_offload_dir: str | None = None

source

disk_offload_max_gb

disk_offload_max_gb: float | None = None

source

host_offload_max_gb

host_offload_max_gb: float | None = None

source

host_staging_alloc

host_staging_alloc: Literal['registered', 'driver'] = 'registered'

source

host_staging_mlock

host_staging_mlock: bool = False

source

num_disk_workers

num_disk_workers: int = 32

source

num_h2d_onload_lanes

num_h2d_onload_lanes: int = 1

source

type

type: KVConnectorType = 'null'

source

use_disk_direct_io

use_disk_direct_io: bool = False

source