For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ModuleV3DecoderPipelineModel

ModuleV3DecoderPipelineModel

class max.pipelines.lib.module_registration.ModuleV3DecoderPipelineModel(pipeline_config, session, *args, **kwargs)

source

Bases: LogProbabilitiesMixin, ModuleV3PipelineModelWithKVCache[TextContext, Any]

Serves a registered text-generation DECODER, and its VISION.

module_pipeline_model() subclasses it for each registration. The decoder is constructed as decoder_cls(config) on a mesh over the pipeline’s devices: a dp axis under data parallelism, else a tp axis. A VISION module runs on the first device; the vision cache copies its embeddings to every device.

Parameters:

• pipeline_config (Any)
• session (InferenceSession)
• args (Any)
• kwargs (Any)

batch_processor_cls

batch_processor_cls

source

alias of DecoderBatchProcessor

decoder_cls

decoder_cls: ClassVar[type[Module[..., Any]]]

source

The registered DECODER class.

ep_comm_initializer

ep_comm_initializer: ExpertParallelExchange | None = None

source

The expert-parallel buffers the decoder takes, under expert parallelism.

execute()

execute(model_inputs)

source

Runs the decoder on the prepared batch.

The batch holds one buffer per device for a distributed input, such as the replicated image embeddings, which is how a captured device graph replays it too, so the decoder runs on the raw buffers.

Parameters:

model_inputs (ModelInputs)

Return type:

ModelOutputs

export_records

export_records: Mapping[str, object] = mappingproxy({})

source

What the completed config records for exporters, by name (export_records()).

graph_specs()

graph_specs(model_config)

source

Returns the decoder graph, and the vision graph when registered.

Under a pipeline stage the decoder graph is the stage’s: after the first it also takes what the previous stage returned, and it binds only its own layers’ weights.

Raises:

• NotImplementedError – If it serves LoRA adapters under data parallelism.
• ValueError – If the config carries a pipeline stage the decoder doesn’t build, or a stage that only traces hands the next a tensor on several devices.
• StageTraced – On a stage that only traces, carrying the types of what it hands the next stage, in the tree it returns them.

Parameters:

model_config (Any)

Return type:

dict[str, GraphSpec]

load_graphs()

load_graphs(session)

source

Compiles the graphs and binds the vision one to the batching.

It keeps session for expert parallelism, which initializes while the config is completed.

Parameters:

session (InferenceSession)

Return type:

dict[str, Model]

pop_video_metrics()

pop_video_metrics()

source

Returns and clears the batch processor’s video encoder metrics.

The pipeline reads video metrics only from the pipeline model, and a registered batch processor that encodes videos counts them.

Return type:

VideoEncoderMetrics | None

pop_vision_metrics()

pop_vision_metrics()

source

Returns and clears the batch processor’s image encoder metrics.

Return type:

VisionEncoderMetrics | None

vision_cls

vision_cls: ClassVar[type[Module[..., Any]] | None] = None

source

The registered VISION class, when the registration has one.

vision_encoding

property vision_encoding: VisionBatchProcessor | None

source

What encodes the batch’s images, when VISION is registered.

It is the registered batch processor.

Raises:

TypeError – If the registered batch processor can’t encode images.