For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python function
cache_dtype_for_encoding
cache_dtype_for_encoding()β
max.pipelines.kv_cache.cache_dtype_for_encoding(quantization_encoding, kv_cache_format)
Returns the KV-cache dtype for a quantization encoding.
An explicit kv_cache_format override takes precedence; otherwise the
dtype is derived from quantization_encoding (float32 when unset).
-
Parameters:
-
- quantization_encoding (SupportedEncoding | None) β The resolved weight encoding, or
None. - kv_cache_format (str | None) β An explicit override string, or
None.
- quantization_encoding (SupportedEncoding | None) β The resolved weight encoding, or
-
Returns:
-
The KV-cache
DType. -
Raises:
-
ValueError β If the override string or encoding is unrecognized.
-
Return type: