For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

decoder_input_types

decoder_input_types()

max.pipelines.lib.module_registration.decoder_input_types(kv_params, device_ref, *, data_parallel_degree=1, vision=None, token_positions=None, batch_context_lengths=None, ep=None, lora=(), constants=None, state=(), scratch=(), input_layout=None)

source

Returns a DECODER graph’s input types, keyed by field.

The keys are the DecoderInputs fields the deployment sets, in its order, and the leaves are the graph’s inputs. The graph takes the tree as its one argument, so an exported graph names its inputs. It takes vision as a VisionMerge group, the kind a serving engine feeds image embeddings by.

Parameters:

• kv_params (KVCacheParamInterface) – The KV cache parameters that supply the kv subtree.
• device_ref (DeviceRef) – The device that holds the token and row-offset tensors.
• data_parallel_degree (int) – The number of replicas. Above one, the graph also takes host_input_row_offsets and data_parallel_splits.
• vision (VisionInputs[TensorLayout] | VisionInputs[TensorType] | None) – The image embeddings’ input types, when the graph is built with a VISION module. A graph built with them takes them on every batch, empty when the batch has no images.
• token_positions (TensorType | None) – The absolute position of each ragged token.
• batch_context_lengths (Sequence[TensorType] | None) – Each replica’s context-length input type, when the decoder reads them.
• ep (object | None) – The expert-parallel input types, when serving with expert parallelism, in the layout of the config’s expert_parallel_inputs.
• lora (Sequence[TensorType]) – The LoRA input types, when serving LoRA adapters.
• constants (object | None) – The checkpoint-backed deployment input types.
• state (Sequence[Any]) – The input types of the decoder’s per-request state.
• scratch (Sequence[BufferType]) – The decoder’s workspace buffer types.
• input_layout (Sequence[str] | None) – The semantic fields in an existing graph’s input order. Defaults to the role’s canonical order.

Return type:

DecoderGraphInputs