IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

PipelineArgs

PipelineArgs​

class max.pipelines.PipelineArgs(*, models=None, config_file=None, section_name=None, model_override=<factory>, task=PipelineTask.UNDEFINED, debug_verify_replay=False, model_path='', served_model_name=None, weight_path=<factory>, quantization_encoding=None, huggingface_model_revision='main', huggingface_weight_revision='main', trust_remote_code=False, subfolder=None, device_specs=<factory>, force_download=False, vision_config_overrides=<factory>, rope_type=None, sliding_window=None, enable_echo=False, chat_template=None, use_subgraphs=True, data_parallel_degree=1, pool_embeddings=True, max_length=None, kv_cache=<factory>, pipeline_role='prefill_and_decode', max_batch_size=None, precompiled_mefs=None, export_mefs=None, max_queue_size_tg=None, min_batch_size_tg=None, ep_size=1, ep_use_allreduce=False, eplb_profile=<factory>, ce_delay_ms=0.0, enable_prioritize_first_decode=False, enable_chunked_prefill=True, chunked_prefill_min_chunk_size=0, enable_in_flight_batching=False, eplb_replicas_per_gpu=0, max_batch_input_tokens=8192, use_experimental_kernels='false', use_vendor_blas='false', use_vendor_ccl='false', custom_architectures=<factory>, execute_empty_batches=False, max_batch_total_tokens=None, device_graph_capture=None, fold_sampler_into_graph=True, max_pending_futures=1, force=False, decode_stall_timeout_s=None, decode_request_ttl_s=None, enable_overlap_scheduler=False, dp_ce_balance_timeout_ms=-1.0, dp_ce_balance_threshold=0.8, dp_ce_balance_enable_dynamic_chunk_size=False, allow_unsupported_logprobs=False, allow_extra_request_fields=False, prefer_module_v3=False, reasoning_parser=None, tool_parser=None, emit_reasoning_content=False, temperature=None, thinking_temperature=None, max_vision_cache_entries=256, denoising_cache=<factory>, in_dtype=float32, out_dtype=float32, enable_structured_output=False, structured_output_backend=None, enable_variable_logits=False, enable_penalties=False, enable_min_tokens=False, sample_on_host=False, gpu_profiling='off', lora=None, speculative=None, draft_model=None)

source

Bases: ConfigFileModel

Flat, user-settable input arguments for a pipeline.

PipelineArgs is the user-facing input to the pipeline system. It holds only scalar fields and a small number of cohesive sub-config objects (kv_cache, lora, speculative, draft_model).

Multi-component pipelines (e.g. diffusion) that require a pre-built ModelManifest may pass models=<manifest> to the constructor. That manifest is stored as a private override and used verbatim by PipelineConfig.from_args() instead of constructing one from the flat scalar fields.

Call PipelineConfig.from_args() to obtain a fully-constructed PipelineConfig ready for architecture-driven resolution.

Parameters:

  • models (ModelManifest | None)
  • config_file (str | None)
  • section_name (str | None)
  • model_override (list[str])
  • task (PipelineTask)
  • debug_verify_replay (bool)
  • model_path (str)
  • served_model_name (str | None)
  • weight_path (list[Path])
  • quantization_encoding (Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'gptq'] | None)
  • huggingface_model_revision (str)
  • huggingface_weight_revision (str)
  • trust_remote_code (bool)
  • subfolder (str | None)
  • device_specs (list[DeviceSpec])
  • force_download (bool)
  • vision_config_overrides (dict[str, Any])
  • rope_type (Literal['none', 'normal', 'neox', 'longrope', 'yarn'] | None)
  • sliding_window (int | None)
  • enable_echo (bool)
  • chat_template (Path | None)
  • use_subgraphs (bool)
  • data_parallel_degree (int)
  • pool_embeddings (bool)
  • max_length (int | None)
  • kv_cache (KVCacheConfig)
  • pipeline_role (Literal['prefill_and_decode', 'prefill_only', 'decode_only'])
  • max_batch_size (int | None)
  • precompiled_mefs (str | None)
  • export_mefs (str | None)
  • max_queue_size_tg (int | None)
  • min_batch_size_tg (int | None)
  • ep_size (int)
  • ep_use_allreduce (bool)
  • eplb_profile (bool)
  • ce_delay_ms (float)
  • enable_prioritize_first_decode (bool)
  • enable_chunked_prefill (bool)
  • chunked_prefill_min_chunk_size (Annotated[int, Ge(ge=0)])
  • enable_in_flight_batching (bool)
  • eplb_replicas_per_gpu (int)
  • max_batch_input_tokens (int)
  • use_experimental_kernels (str)
  • use_vendor_blas (str)
  • use_vendor_ccl (str)
  • custom_architectures (list[str])
  • execute_empty_batches (bool)
  • max_batch_total_tokens (int | None)
  • device_graph_capture (bool | None)
  • fold_sampler_into_graph (bool)
  • max_pending_futures (int)
  • force (bool)
  • decode_stall_timeout_s (float | None)
  • decode_request_ttl_s (float | None)
  • enable_overlap_scheduler (bool)
  • dp_ce_balance_timeout_ms (float)
  • dp_ce_balance_threshold (float)
  • dp_ce_balance_enable_dynamic_chunk_size (bool)
  • allow_unsupported_logprobs (bool)
  • allow_extra_request_fields (bool)
  • prefer_module_v3 (bool)
  • reasoning_parser (str | None)
  • tool_parser (str | None)
  • emit_reasoning_content (bool)
  • temperature (float | None)
  • thinking_temperature (float | None)
  • max_vision_cache_entries (int)
  • denoising_cache (DenoisingCacheConfig)
  • in_dtype (DType)
  • out_dtype (DType)
  • enable_structured_output (bool)
  • structured_output_backend (str | None)
  • enable_variable_logits (bool)
  • enable_penalties (bool)
  • enable_min_tokens (bool)
  • sample_on_host (bool)
  • gpu_profiling (Literal['off', 'on', 'detailed'])
  • lora (LoRAConfig | None)
  • speculative (SpeculativeConfig | None)
  • draft_model (MAXModelConfig | None)

allow_extra_request_fields​

allow_extra_request_fields: bool

source

allow_unsupported_logprobs​

allow_unsupported_logprobs: bool

source

ce_delay_ms​

ce_delay_ms: float

source

chat_template​

chat_template: Path | None

source

chunked_prefill_min_chunk_size​

chunked_prefill_min_chunk_size: int

source

custom_architectures​

custom_architectures: list[str]

source

data_parallel_degree​

data_parallel_degree: int

source

debug_verify_replay​

debug_verify_replay: bool

source

decode_request_ttl_s​

decode_request_ttl_s: float | None

source

decode_stall_timeout_s​

decode_stall_timeout_s: float | None

source

denoising_cache​

denoising_cache: DenoisingCacheConfig

source

device_graph_capture​

device_graph_capture: bool | None

source

device_specs​

device_specs: list[DeviceSpec]

source

dp_ce_balance_enable_dynamic_chunk_size​

dp_ce_balance_enable_dynamic_chunk_size: bool

source

dp_ce_balance_threshold​

dp_ce_balance_threshold: float

source

dp_ce_balance_timeout_ms​

dp_ce_balance_timeout_ms: float

source

draft_model​

draft_model: MAXModelConfig | None

source

emit_reasoning_content​

emit_reasoning_content: bool

source

enable_chunked_prefill​

enable_chunked_prefill: bool

source

enable_echo​

enable_echo: bool

source

enable_in_flight_batching​

enable_in_flight_batching: bool

source

enable_min_tokens​

enable_min_tokens: bool

source

enable_overlap_scheduler​

enable_overlap_scheduler: bool

source

enable_penalties​

enable_penalties: bool

source

enable_prioritize_first_decode​

enable_prioritize_first_decode: bool

source

enable_structured_output​

enable_structured_output: bool

source

enable_variable_logits​

enable_variable_logits: bool

source

ep_size​

ep_size: int

source

ep_use_allreduce​

ep_use_allreduce: bool

source

eplb_profile​

eplb_profile: bool

source

eplb_replicas_per_gpu​

eplb_replicas_per_gpu: int

source

execute_empty_batches​

execute_empty_batches: bool

source

export_mefs​

export_mefs: str | None

source

fold_sampler_into_graph​

fold_sampler_into_graph: bool

source

force​

force: bool

source

force_download​

force_download: bool

source

from_flat_kwargs()​

classmethod from_flat_kwargs(**kwargs)

source

Construct a PipelineArgs from a flat CLI kwargs namespace.

Routes flat kwargs (the same format accepted by PipelineConfig.from_flat_kwargs()) into the appropriate flat fields of PipelineArgs. Delegates to PipelineConfig.from_flat_kwargs() for the heavy-lifting of routing and sub-config construction, then extracts the user-facing fields.

Parameters:

**kwargs (Any) – Flat keyword arguments, e.g. model_path, kv_cache_size, enable_lora.

Returns:

A PipelineArgs populated from the flat kwargs.

Return type:

Self

from_pipeline_config()​

classmethod from_pipeline_config(pipeline_config)

source

Construct a PipelineArgs from an existing PipelineConfig.

Extracts the user-facing flat fields from a PipelineConfig and returns a PipelineArgs populated from them.

This exists to let from_flat_kwargs() reuse PipelineConfig.from_flat_kwargs()’s flat-kwarg routing logic (parsing --model-override, building the draft model config, etc.) instead of duplicating it. It is not a general round-trip: pipeline_config is expected to be freshly constructed and not yet resolved. Resolution-derived state (e.g. an applied dtype cast recorded during architecture-level resolution) is not preserved – PipelineArgs is deliberately isolated from resolution mutations (see #90128), so passing an already-resolved pipeline_config here will silently drop that state.

Parameters:

pipeline_config (PipelineConfig) – The source PipelineConfig to extract from. Should not have had PipelineConfig.resolve() called on it.

Returns:

A PipelineArgs populated from the given config.

Return type:

Self

gpu_profiling​

gpu_profiling: GPUProfilingMode

source

huggingface_model_revision​

huggingface_model_revision: str

source

huggingface_weight_revision​

huggingface_weight_revision: str

source

in_dtype​

in_dtype: DType

source

kv_cache​

kv_cache: KVCacheConfig

source

lora​

lora: LoRAConfig | None

source

main_architecture_name​

property main_architecture_name: str

source

Returns the HuggingFace architecture class name for the main model.

Reads architectures[0] from the model’s HuggingFace config without constructing a full PipelineConfig.

Raises:

ValueError – If the architecture name cannot be determined.

max_batch_input_tokens​

max_batch_input_tokens: int

source

max_batch_size​

max_batch_size: int | None

source

max_batch_total_tokens​

max_batch_total_tokens: int | None

source

max_length​

max_length: int | None

source

max_pending_futures​

max_pending_futures: int

source

max_queue_size_tg​

max_queue_size_tg: int | None

source

max_vision_cache_entries​

max_vision_cache_entries: int

source

min_batch_size_tg​

min_batch_size_tg: int | None

source

model_config​

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_override​

model_override: list[str]

source

model_path​

model_path: str

source

model_post_init()​

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self (BaseModel) – The BaseModel instance.
  • context (Any) – The context.

Return type:

None

out_dtype​

out_dtype: DType

source

pipeline_role​

pipeline_role: PipelineRole

source

pool_embeddings​

pool_embeddings: bool

source

precompiled_mefs​

precompiled_mefs: str | None

source

prefer_module_v3​

prefer_module_v3: bool

source

quantization_encoding​

quantization_encoding: SupportedEncoding | None

source

reasoning_parser​

reasoning_parser: str | None

source

rope_type​

rope_type: RopeType | None

source

sample_on_host​

sample_on_host: bool

source

served_model_name​

served_model_name: str | None

source

sliding_window​

sliding_window: int | None

source

speculative​

speculative: SpeculativeConfig | None

source

structured_output_backend​

structured_output_backend: str | None

source

subfolder​

subfolder: str | None

source

task​

task: PipelineTask

source

temperature​

temperature: float | None

source

thinking_temperature​

thinking_temperature: float | None

source

tool_parser​

tool_parser: str | None

source

trust_remote_code​

trust_remote_code: bool

source

use_experimental_kernels​

use_experimental_kernels: str

source

use_subgraphs​

use_subgraphs: bool

source

use_vendor_blas​

use_vendor_blas: str

source

use_vendor_ccl​

use_vendor_ccl: str

source

vision_config_overrides​

vision_config_overrides: dict[str, Any]

source

weight_path​

weight_path: list[Path]

source

Models

Cookbook

Puzzles

Blog

Community

Contact