For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

SingleReplicaRaggedBatchProcessor

SingleReplicaRaggedBatchProcessor

class max.pipelines.lib.interfaces.SingleReplicaRaggedBatchProcessor(config, runtime, inputs=())

source

Bases: RaggedBatchProcessor[ContextT, InputsT, ArchConfigT_co]

Single-replica ragged KV batching for Graph-path models (no DP / LoRA).

Subclasses implement _make_inputs() to construct their architecture- specific ModelInputs type. Override _include_signal_buffers or get_symbolic_inputs() when signal-buffer wiring differs.

Parameters:

• config (ArchConfigT_co)
• runtime (BatchProcessorRuntime)
• inputs (Sequence[InputDescriptor])

get_symbolic_inputs()

get_symbolic_inputs(*, kv_params, device_refs)

source

Returns symbolic graph inputs for single-replica ragged KV models.

Parameters:

• kv_params (KVCacheParamInterface)
• device_refs (list[DeviceRef])

Return type:

list[TensorType | BufferType]

prepare_initial_token_inputs()

prepare_initial_token_inputs(replica_batches, kv_cache_inputs=None, return_n_logits=1)

source

Prepares ragged token inputs for a single-replica Graph-path batch.

Parameters:

• replica_batches (Sequence[Sequence[ContextT]])
• kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
• return_n_logits (int)

Return type:

InputsT

process_outputs()

process_outputs(outputs)

source

Maps raw execution buffers to ModelOutputs.

Parameters:

outputs (Sequence[Buffer | object])

Return type:

ModelOutputs