For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
GraphApiDecoderModelInputs
GraphApiDecoderModelInputs
class max.pipelines.lib.registration_graph_api.GraphApiDecoderModelInputs(*, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, tokens, input_row_offsets, return_n_logits, token_positions=None, host_input_row_offsets=None, data_parallel_splits=None, batch_context_lengths=None, ep_inputs=(), signal_buffers=(), constants=None, state=(), scratch=(), stage_boundary=None, request_ids=(), input_tree_def=None, batch_fields=('tokens', 'input_row_offsets', 'return_n_logits'))
Bases: DecoderModelInputs
The prepared inputs of a registered graph-API DECODER.
-
Parameters:
-
- kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
- lora_buffers (tuple[Buffer, ...])
- vision_embeddings (list[Buffer])
- vision_scatter_indices (list[Buffer])
- hidden_states (Buffer | list[Buffer] | None)
- tokens (Buffer)
- input_row_offsets (Buffer)
- return_n_logits (Buffer)
- token_positions (Buffer | None)
- host_input_row_offsets (Buffer | None)
- data_parallel_splits (Buffer | None)
- batch_context_lengths (list[Buffer] | None)
- ep_inputs (tuple[Buffer, ...])
- signal_buffers (tuple[Buffer, ...])
- constants (tree.Tree[Buffer] | None)
- state (tuple[Buffer, ...])
- scratch (tuple[Buffer, ...])
- stage_boundary (tree.Tree[Buffer] | None)
- request_ids (tuple[RequestID, ...])
- input_tree_def (tree.TreeDef | None)
- batch_fields (tuple[str, ...])
batch_fields
batch_fields: tuple[str, ...] = ('tokens', 'input_row_offsets', 'return_n_logits')
The batch fields the graph takes, in its order.
buffers
The step’s buffers, in the graph’s input order.