For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.lib.registration_graph_api

Serves architectures that register a graph-API module by role.

A graph-API (Module) DECODER registers like a ModuleV3 one: registered_pipeline_model() builds the pipeline model, which finalizes the config, constructs the decoder as decoder_cls(config), loads the checkpoint with load_state_dict(), builds the graph from graph_api_decoder_input_types(), and batches and executes it. The decoder’s __call__ takes DecoderInputs of graph values and returns DecoderOutputs.

A registered graph-API decoder class can also set:

• graph_name, the name its graph compiles under (graph_name()), which an instance may also set from its config;
• supports_data_parallel, when it runs one replica per device and reads data_parallel_splits;
• data_parallel_batch_inputs, the DecoderInputs batch fields its data-parallel graph takes, in that graph’s order, when they differ from the canonical ones (graph_api_decoder_input_types());
• always_signal_buffers, when it takes allreduce signal buffers even on one device;
• strict_state_dict_loading, False when the checkpoint may hold weights the decoder doesn’t read;
• override_quantization_encoding, when the checkpoint’s quantization encoding overrides the one a weight declares;
• build_module(context, config_cls), a classmethod that builds the module a speculative-decoding graph calls as its target. The pipeline model then exposes it as its own build_module;
• state_store(config, *, max_batch_size, devices), a classmethod that returns the store of the per-request state it carries between steps (see request_state). Its batch processor must be a DecoderBatchProcessor, which feeds the state.

Pipeline models

GraphApiDecoderPipelineModel	Serves a registered graph-API DECODER module.

graph_api_pipeline_model	Returns the pipeline model that serves a graph-API registration.
registered_pipeline_model	Returns the pipeline model that serves a module registration.

Batch processors

GraphApiDecoderBatchProcessor	Batches ragged text for a registered graph-API DECODER.

Model inputs

GraphApiDecoderModelInputs	The prepared inputs of a registered graph-API DECODER.

Module loading

is_graph_api_module	Returns whether module_cls is a graph-API module class.
load_graph_api_module	Constructs a registered module as module_cls(config) and loads it.

Graph input types and outputs

decoder_outputs	Returns a graph-API text model's grouped outputs as the role's.
graph_api_decoder_input_types	Returns a graph-API DECODER graph's input types.