IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.unified_mtp_qwen3_5

Qwen3.5 MTP head fused with its target for speculative decoding.

UnifiedMTPQwen3_5Inputs​

class max.pipelines.architectures.unified_mtp_qwen3_5.UnifiedMTPQwen3_5Inputs(request_ids=None, decoder_position_ids=None, vision_inputs_in_graph=True, *, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, tokens, input_row_offsets, signal_buffers, return_n_logits, data_parallel_splits=None, draft_tokens=None, draft_probs_full=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, draft_slot_ids=None, draft_block_offsets=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False, host_input_row_offsets, batch_context_lengths, live_conv_pools, live_recurrent_pools, live_conv_row_ids, live_recurrent_row_ids, ring_pools, ring_row_ids, position_ids=None)

source

Bases: UnifiedSpecDecodeInputs, Qwen3_5Inputs

Inputs for the fused Qwen3.5 MTP graph.

The prefix and the spec-decode tail follow the canonical unified ordering; everything after the bitmask triple is this architecture’s state-pool tail, which no other unified MTP graph has. Also a Qwen3_5Inputs, so the Qwen3.5 model and batch processor overrides keep their return types.

Parameters:

batch_context_lengths​

batch_context_lengths: list[Buffer]

source

buffers​

property buffers: tuple[Buffer, ...]

source

Returns positional Buffer inputs for model ABI calls.

host_input_row_offsets​

host_input_row_offsets: Buffer

source

input_row_offsets​

input_row_offsets: Buffer

source

Tensor containing the offsets for each row in the ragged input sequence.

live_conv_pools​

live_conv_pools: list[Buffer]

source

live_conv_row_ids​

live_conv_row_ids: list[Buffer]

source

live_recurrent_pools​

live_recurrent_pools: list[Buffer]

source

live_recurrent_row_ids​

live_recurrent_row_ids: list[Buffer]

source

position_ids​

position_ids: Buffer | None = None

source

[3, merged_total_seq_len] M-RoPE positions for the merged [real, draft_1..draft_k] window. None on a text-only graph, whose rotary stays on the static cache-derived table.

return_n_logits​

return_n_logits: Buffer

source

ring_pools​

ring_pools: list[Buffer]

source

ring_row_ids​

ring_row_ids: list[Buffer]

source

The ring’s [num_layers, batch_size] rows per device.

signal_buffers​

signal_buffers: list[Buffer]

source

Device buffers used for synchronization in communication collectives.

tokens​

tokens: Buffer

source

Tensor containing the input token IDs.

UnifiedMTPQwen3_5Model​

class max.pipelines.architectures.unified_mtp_qwen3_5.UnifiedMTPQwen3_5Model(*args, **kwargs)

source

Bases: _UnifiedSpecDecodeModelMixin, Qwen3_5Model

Qwen3.5 with MTP: merge, verify, roll the state back, and draft.

Parameters:

batch_processor_cls​

batch_processor_cls

source

alias of UnifiedMTPQwen3_5BatchProcessor

load_model()​

load_model(session)

source

Compiles the one fused graph.

Skips the base architecture’s vision encoder, since this graph is text-only.

Parameters:

session (InferenceSession)

Return type:

Model

model_config_cls​

model_config_cls

source

alias of UnifiedMTPQwen3_5Config