For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python function

gptq_quant_config

gptq_quant_config()

max.pipelines.weights.gptq_quant_config(quantization_encoding, huggingface_config)

source

Builds the GPTQ QuantizationConfig for a "gptq" encoding.

Computed on demand from the resolved quantization_encoding and the checkpoint’s Hugging Face quantization_config; returns None for any non-GPTQ encoding.

Parameters:

• quantization_encoding (SupportedEncoding | None) – The resolved weight encoding.
• huggingface_config (AutoConfig) – The checkpoint’s Hugging Face config.

Returns:

The GPTQ QuantizationConfig, or None when the encoding is not GPTQ.

Raises:

ValueError – If the GPTQ scales are not float16.

Return type:

QuantizationConfig | None

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit