For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

PipelineConfig

PipelineConfig

class max.pipelines.PipelineConfig(*, config_file=None, section_name=None, debug_verify_replay=False, models=<factory>, model_override=<factory>, sampling=<factory>, profiling=<factory>, lora=None, speculative=None, runtime=<factory>, task=PipelineTask.UNDEFINED)

source

Bases: ConfigFileModel

Configuration for a pipeline.

Contains settings for model selection, batch sizing, sampling, profiling, LoRA adapters, and speculative decoding. Once initialized, all fields are resolved to their final values from CLI flags, config files, environment variables, or internal defaults.

Parameters:

• config_file (str | None)
• section_name (str | None)
• debug_verify_replay (bool)
• models (dict[str, MAXModelConfig])
• model_override (list[str])
• sampling (SamplingConfig)
• profiling (ProfilingConfig)
• lora (LoRAConfig | None)
• speculative (SpeculativeConfig | None)
• runtime (PipelineRuntimeConfig)
• task (PipelineTask)

configure_session()

configure_session(session)

source

Configures a InferenceSession with standard pipeline settings.

Parameters:

session (InferenceSession)

Return type:

None

debug_verify_replay

debug_verify_replay: bool

source

Whether to run eager verification before device graph replay.

draft_model

property draft_model: MAXModelConfig | None

source

The draft model configuration. Alias for models.get("draft").

estimate_signal_buffer_memory()

estimate_signal_buffer_memory(arch_config=None)

source

Estimates total signal-buffer memory across all devices.

Signal buffers are fixed-size (NUM_BYTES) per-GPU allocations used by P2P collectives. The only site visible from PipelineConfig is the main model graph, and only for multi-GPU pipelines. The tiered/rust_tiered KV connectors fan MLA-replicated blocks out via plain P2P copies, not a signal-buffer broadcast (see rust_kv/kv-tier-connector/src/copy_engine.rs), so they contribute no additional term here.

Returns 0 for single-device pipelines.

Parameters:

arch_config (ArchConfig | None) – Unused; kept for interface parity with MemoryPlanner.estimate_signal_buffer_memory().

Returns:

Estimated total signal-buffer memory in bytes (across all devices).

Return type:

int

from_args()

classmethod from_args(args)

source

Construct a PipelineConfig from a PipelineArgs.

Resolution runs before construction: the architecture is looked up from the args’ models, every architecture-dependent value is computed as plain data, and the config is constructed exactly once, already carrying its final values. args is the read-only input record throughout.

Parameters:

args (PipelineArgs) – Flat user-facing pipeline arguments.

Returns:

A fully constructed and validated PipelineConfig.

Return type:

Self

lora

lora: LoRAConfig | None

source

The LoRA configuration.

model

property model: MAXModelConfig

source

The main model config. Alias for models["main"].

model_config

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True, 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_override

model_override: list[str]

source

Per-component model overrides applied before resolution.

model_post_init()

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.
• context (Any) – The context.

Return type:

None

models

models: _ModelsType

source

The model manifest containing all model configs keyed by role.

needs_bitmask_constraints

property needs_bitmask_constraints: bool

source

Whether constrained decoding can fire and requires the bitmask path.

True if the user enabled --enable-structured-output (for user-supplied response_format=json_schema) or a tool parser is configured (tool-call grammars work without the flag — they are server-generated and gated on having a parser that can both produce the grammar and parse the resulting output).

Tool-call constrained decoding can be turned off independently via sampling.enable_tool_call_constrained_decode: when that is False the tool parser still parses tool calls out of generated text, but no grammar is generated and the bitmask path is not needed on its account.

Drives whether model / sampler graphs are compiled with a bitmask input and whether the D2H pinned buffer is allocated. Distinct from sampling.enable_structured_output, which is the user-facing flag and only gates honoring user-supplied JSON schemas.

profiling

profiling: ProfilingConfig

source

The profiling configuration.

runtime

runtime: PipelineRuntimeConfig

source

The model-agnostic runtime settings for pipeline execution.

sampling

sampling: SamplingConfig

source

The sampling configuration.

speculative

speculative: SpeculativeConfig | None

source

The speculative decoding configuration.

task

task: PipelineTask

source

The pipeline task, used for arch disambiguation during config resolution.