For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

PipelineConfig

PipelineConfig

class max.pipelines.PipelineConfig(*, config_file=None, section_name=None, debug_verify_replay=False, models=<factory>, model_override=<factory>, sampling=<factory>, profiling=<factory>, lora=None, speculative=None, runtime=<factory>, task=PipelineTask.UNDEFINED)

source

Bases: ConfigFileModel

Configuration for a pipeline.

Contains settings for model selection, batch sizing, sampling, profiling, LoRA adapters, and speculative decoding. Once initialized, all fields are resolved to their final values from CLI flags, config files, environment variables, or internal defaults.

Parameters:

• config_file (str | None)
• section_name (str | None)
• debug_verify_replay (bool)
• models (dict[str, MAXModelConfig])
• model_override (list[str])
• sampling (SamplingConfig)
• profiling (ProfilingConfig)
• lora (LoRAConfig | None)
• speculative (SpeculativeConfig | None)
• runtime (PipelineRuntimeConfig)
• task (PipelineTask)

configure_session()

configure_session(session)

source

Configures a InferenceSession with standard pipeline settings.

Parameters:

session (InferenceSession)

Return type:

None

debug_verify_replay

debug_verify_replay: bool

source

Whether to run eager verification before device graph replay.

draft_model

property draft_model: MAXModelConfig | None

source

The draft model configuration. Alias for models.get("draft").

estimate_signal_buffer_memory()

estimate_signal_buffer_memory(arch_config=None)

source

Estimates total signal-buffer memory across all devices.

Signal buffers are fixed-size (NUM_BYTES) per-GPU allocations used by P2P collectives. Each independent allocation site contributes one set of ngpus buffers. The base estimate counts the sites visible from PipelineConfig:

• main model graph (multi-GPU only),
• BlockOffloadEngine for KV-cache offloading, only when its replicate_kv_across_tp path is active (MLA model with DP=1 and multi-device TP). See block_copy_engine.py / transfer_engine.py.

Returns 0 for single-device pipelines.

Parameters:

arch_config (ArchConfig | None) – Optional architecture config. When provided and it exposes KV params, the BCE term is gated on the actual replicates_kv_across_tp flag rather than only the kv_connector setting. Without it, the BCE term is added whenever a connector is configured (conservative).

Returns:

Estimated total signal-buffer memory in bytes (across all devices).

Return type:

int

from_args()

classmethod from_args(args)

source

Construct a PipelineConfig from a PipelineArgs.

Parameters:

args (PipelineArgs) – Flat user-facing pipeline arguments.

Returns:

A fully constructed PipelineConfig ready for architecture-driven resolution via resolve().

Return type:

Self

from_flat_kwargs()

classmethod from_flat_kwargs(**kwargs)

source

Construct a PipelineConfig from a flat CLI kwargs namespace.

Accepts the flat kwargs produced by pipeline_config_options (for example model_path, kv_cache_size, enable_lora) and routes them into the appropriate sub-configs before constructing the instance.

This is the entry point for CLI and legacy callers. Direct construction via PipelineConfig(models=..., runtime=..., ...) with properly typed sub-configs is also supported and requires no routing.

Parameters:

kwargs (Any)

Return type:

Self

lora

lora: LoRAConfig | None

source

The LoRA configuration.

model

property model: MAXModelConfig

source

The main model config. Alias for models["main"].

model_config

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_override

model_override: list[str]

source

Per-component model overrides applied before resolution.

model_post_init()

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.
• context (Any) – The context.

Return type:

None

models

models: _ModelsType

source

The model manifest containing all model configs keyed by role.

needs_bitmask_constraints

property needs_bitmask_constraints: bool

source

Whether constrained decoding can fire and requires the bitmask path.

True if the user enabled --enable-structured-output (for user-supplied response_format=json_schema) or a tool parser is configured (tool-call grammars work without the flag — they are server-generated and gated on having a parser that can both produce the grammar and parse the resulting output).

Tool-call constrained decoding can be turned off independently via sampling.enable_tool_call_constrained_decode: when that is False the tool parser still parses tool calls out of generated text, but no grammar is generated and the bitmask path is not needed on its account.

Drives whether model / sampler graphs are compiled with a bitmask input and whether the D2H pinned buffer is allocated. Distinct from sampling.enable_structured_output, which is the user-facing flag and only gates honoring user-supplied JSON schemas.

profiling

profiling: ProfilingConfig

source

The profiling configuration.

resolve()

resolve(arch, draft_arch=None)

source

Validates the config.

Parameters:

• arch (Any) – Pre-resolved target architecture from the registry.
• draft_arch (Any) – Pre-resolved draft architecture (speculative decoding only). Required when draft_model is set.

Return type:

None

runtime

runtime: PipelineRuntimeConfig

source

The model-agnostic runtime settings for pipeline execution.

sampling

sampling: SamplingConfig

source

The sampling configuration.

speculative

speculative: SpeculativeConfig | None

source

The speculative decoding configuration.

task

task: PipelineTask

source

The pipeline task, used for arch disambiguation during config resolution.