For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ByteCount

ByteCount

class max.pipelines.kv_cache.ByteCount(free, total)

source

Bases: object

A point-in-time snapshot of an external cache tier’s occupancy, in bytes.

Bytes rather than blocks because a connector’s host and disk tiers are byte budgets that the operator sizes in bytes (host_offload_max_gb, disk_offload_max_gb), and because their block width need not match the device’s – an MLA-replicated unit is stored once on the host and broadcast back on load. Bytes also rate directly against PCIe and disk bandwidth.

Parameters:

• free (int)
• total (int)

free

free: int

source

free_pct

property free_pct: float

source

Returns the percentage of bytes not currently in use, in [0, 100].

0 when total is 0, rather than dividing by zero.

total

total: int

source

used

property used: int

source

Returns the bytes currently in use.

used_pct

property used_pct: float

source

Returns the percentage of bytes currently in use, in [0, 100].

0 when total is 0 (e.g. a tier with no configured capacity), rather than dividing by zero.