For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

BatchProcessor

BatchProcessor

class max.pipelines.lib.interfaces.BatchProcessor(config, runtime, inputs=())

source

Bases: ABC, Generic[ContextT, InputsT, ArchConfigT_co]

Batches pipeline contexts into model inputs and parses execution outputs.

Parameters:

• config (ArchConfigT_co)
• runtime (BatchProcessorRuntime)
• inputs (Sequence[InputDescriptor])

bind_input_tree()

bind_input_tree(input_tree)

source

Receives the named inputs of the graph this processor feeds.

The model owns the tree and builds its graph from it, so the tree cannot describe a different graph than the one the processor feeds.

Parameters:

input_tree (OrderedDict[str, object]) – The graph’s inputs keyed by name, whose leaves are the graph’s input types in declaration order.

Return type:

None

get_symbolic_inputs()

abstract get_symbolic_inputs(*, kv_params, device_refs)

source

Returns non-KV graph input types in execution order.

Parameters:

• kv_params (KVCacheParamInterface)
• device_refs (list[DeviceRef])

Return type:

list[TensorType | BufferType]

input_tree

input_tree: OrderedDict[str, object] | None = None

source

The graph’s named inputs, or None until the model binds them.

prepare_initial_token_inputs()

abstract prepare_initial_token_inputs(replica_batches, kv_cache_inputs=None, return_n_logits=1)

source

Prepares inputs for the first execution step of a batch.

Parameters:

• replica_batches (Sequence[Sequence[ContextT]])
• kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
• return_n_logits (int)

Return type:

InputsT

process_outputs()

abstract process_outputs(outputs)

source

Maps raw Model.execute buffers to ModelOutputs.

Parameters:

outputs (Sequence[Buffer | object])

Return type:

ModelOutputs