For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.architectures.unified_mtp_qwen3_5
Qwen3.5 MTP head fused with its target for speculative decoding.
UnifiedMTPQwen3_5Inputs
class max.pipelines.architectures.unified_mtp_qwen3_5.UnifiedMTPQwen3_5Inputs(request_ids=None, decoder_position_ids=None, vision_inputs_in_graph=True, *, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, tokens, input_row_offsets, signal_buffers, return_n_logits, data_parallel_splits=None, draft_tokens=None, draft_probs_full=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, draft_slot_ids=None, draft_block_offsets=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False, host_input_row_offsets, batch_context_lengths, live_conv_pools, live_recurrent_pools, live_conv_row_ids, live_recurrent_row_ids, ring_pools, ring_row_ids, position_ids=None)
Bases: UnifiedSpecDecodeInputs, Qwen3_5Inputs
Inputs for the fused Qwen3.5 MTP graph.
The prefix and the spec-decode tail follow the canonical unified ordering;
everything after the bitmask triple is this architecture’s state-pool tail,
which no other unified MTP graph has. Also a Qwen3_5Inputs, so
the Qwen3.5 model and batch processor overrides keep their return types.
-
Parameters:
-
- request_ids (list[RequestID] | None)
- decoder_position_ids (Buffer | None)
- vision_inputs_in_graph (bool)
- kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
- lora_buffers (tuple[Buffer, ...])
- vision_embeddings (list[Buffer])
- vision_scatter_indices (list[Buffer])
- hidden_states (Buffer | list[Buffer] | None)
- tokens (Buffer)
- input_row_offsets (Buffer)
- signal_buffers (list[Buffer])
- return_n_logits (Buffer)
- data_parallel_splits (Buffer)
- draft_tokens (Buffer | None)
- draft_probs_full (Buffer | None)
- seed (Buffer | None)
- temperature (Buffer | None)
- top_k (Buffer | None)
- max_k (Buffer | None)
- top_p (Buffer | None)
- min_top_p (Buffer | None)
- in_thinking_phase (Buffer | None)
- draft_slot_ids (Buffer | None)
- draft_block_offsets (Buffer | None)
- pinned_bitmask (Buffer | None)
- wait_payload (Buffer | None)
- device_bitmask_scratch (Buffer | None)
- structured_output (bool)
- host_input_row_offsets (Buffer)
- batch_context_lengths (list[Buffer])
- live_conv_pools (list[Buffer])
- live_recurrent_pools (list[Buffer])
- live_conv_row_ids (list[Buffer])
- live_recurrent_row_ids (list[Buffer])
- ring_pools (list[Buffer])
- ring_row_ids (list[Buffer])
- position_ids (Buffer | None)
batch_context_lengths
buffers
Returns positional Buffer inputs for model ABI calls.
host_input_row_offsets
host_input_row_offsets: Buffer
input_row_offsets
input_row_offsets: Buffer
Tensor containing the offsets for each row in the ragged input sequence.
live_conv_pools
live_conv_row_ids
live_recurrent_pools
live_recurrent_row_ids
position_ids
[3, merged_total_seq_len] M-RoPE positions for the merged
[real, draft_1..draft_k] window. None on a text-only graph,
whose rotary stays on the static cache-derived table.
return_n_logits
return_n_logits: Buffer
ring_pools
ring_row_ids
The ring’s [num_layers, batch_size] rows per device.
signal_buffers
Device buffers used for synchronization in communication collectives.
tokens
tokens: Buffer
Tensor containing the input token IDs.
UnifiedMTPQwen3_5Model
class max.pipelines.architectures.unified_mtp_qwen3_5.UnifiedMTPQwen3_5Model(*args, **kwargs)
Bases: _UnifiedSpecDecodeModelMixin, Qwen3_5Model
Qwen3.5 with MTP: merge, verify, roll the state back, and draft.
batch_processor_cls
batch_processor_cls
alias of UnifiedMTPQwen3_5BatchProcessor
load_model()
load_model(session)
Compiles the one fused graph.
Skips the base architecture’s vision encoder, since this graph is text-only.
-
Parameters:
-
session (InferenceSession)
-
Return type:
model_config_cls
model_config_cls
alias of UnifiedMTPQwen3_5Config