For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

read_modelopt_quantized_layers

read_modelopt_quantized_layers()

max.pipelines.weights.read_modelopt_quantized_layers(hf_quant_config, *, allowed, expected_modules=None, quant_algos=('MIXED_PRECISION',), kv_cache_quant_algos=None, model_name=None)

source

Reads and validates the per-module map of a modelopt checkpoint.

modelopt exports a checkpoint whose modules differ in precision as quant_algo: "MIXED_PRECISION" and records each quantized module’s algorithm in quantized_layers; every module it does not list is unquantized. A module read in the wrong format loads without error and produces wrong logits, so everything the caller has not declared it can read is refused here.

Parameters:

• hf_quant_config (Mapping[str, object]) – The resolved quantization config, from resolve_hf_quant_config().
• allowed (Collection[tuple[str, int | None]]) – The (quant_algo, group_size) pairs the caller can load. A group_size of None matches an entry that declares none.
• expected_modules (Collection[str] | None) – When given, the exact set of module names quantized_layers must list.
• quant_algos (Collection[str]) – The top-level quant_algo values to accept.
• kv_cache_quant_algos (Collection[str | None] | None) – When given, the kv_cache_quant_algo values to accept, with None for an unquantized KV cache.
• model_name (str | None) – The name that prefixes each error message.

Returns:

Each listed module’s quantization, keyed by its checkpoint name.

Raises:

ValueError – If the config is not a modelopt config with an accepted quant_algo and kv_cache_quant_algo, carries no quantized_layers map, lists a module outside expected_modules or misses one of them, or declares an entry outside allowed.

Return type:

dict[str, ModelOptModuleQuant]