For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.unified_mtp_qwen3_5

Qwen3.5 MTP head fused with its target for speculative decoding.

UnifiedMTPQwen3_5Inputs

class max.pipelines.architectures.unified_mtp_qwen3_5.UnifiedMTPQwen3_5Inputs(tokens, input_row_offsets, host_input_row_offsets, return_n_logits, data_parallel_splits, signal_buffers, batch_context_lengths, slot_idx, live_conv_pools, live_recurrent_pools, shadow_conv_pools, shadow_recurrent_pools, *, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, draft_tokens=None, draft_probs_full=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False, sampled_draft_proposal=False)

source

Bases: UnifiedSpecDecodeInputs

Inputs for the fused Qwen3.5 MTP graph.

The prefix and the spec-decode tail follow the canonical unified ordering; everything after the bitmask triple is this architecture’s state-pool tail, which no other unified MTP graph has.

Parameters:

• tokens (Buffer)
• input_row_offsets (Buffer)
• host_input_row_offsets (Buffer)
• return_n_logits (Buffer)
• data_parallel_splits (Buffer)
• signal_buffers (list[Buffer])
• batch_context_lengths (list[Buffer])
• slot_idx (list[Buffer])
• live_conv_pools (list[Buffer])
• live_recurrent_pools (list[Buffer])
• shadow_conv_pools (list[Buffer])
• shadow_recurrent_pools (list[Buffer])
• kv_cache_inputs (KVCacheInputsInterface[Buffer, Buffer] | None)
• lora_buffers (tuple[Buffer, ...])
• vision_embeddings (list[Buffer])
• vision_scatter_indices (list[Buffer])
• hidden_states (Buffer | list[Buffer] | None)
• draft_tokens (Buffer | None)
• draft_probs_full (Buffer | None)
• seed (Buffer | None)
• temperature (Buffer | None)
• top_k (Buffer | None)
• max_k (Buffer | None)
• top_p (Buffer | None)
• min_top_p (Buffer | None)
• in_thinking_phase (Buffer | None)
• pinned_bitmask (Buffer | None)
• wait_payload (Buffer | None)
• device_bitmask_scratch (Buffer | None)
• structured_output (bool)
• sampled_draft_proposal (bool)

batch_context_lengths

batch_context_lengths: list[Buffer]

source

buffers

property buffers: tuple[Buffer, ...]

source

Returns positional Buffer inputs for model ABI calls.

data_parallel_splits

data_parallel_splits: Buffer

source

host_input_row_offsets

host_input_row_offsets: Buffer

source

input_row_offsets

input_row_offsets: Buffer

source

live_conv_pools

live_conv_pools: list[Buffer]

source

live_recurrent_pools

live_recurrent_pools: list[Buffer]

source

return_n_logits

return_n_logits: Buffer

source

shadow_conv_pools

shadow_conv_pools: list[Buffer]

source

shadow_recurrent_pools

shadow_recurrent_pools: list[Buffer]

source

signal_buffers

signal_buffers: list[Buffer]

source

slot_idx

slot_idx: list[Buffer]

source

tokens

tokens: Buffer

source

UnifiedMTPQwen3_5Model

class max.pipelines.architectures.unified_mtp_qwen3_5.UnifiedMTPQwen3_5Model(*args, **kwargs)

source

Bases: _UnifiedSpecDecodeModelMixin, Qwen3_5Model

Qwen3.5 with MTP: merge, verify, roll the state back, and draft.

Parameters:

• args (Any)
• kwargs (Any)

load_model()

load_model(session)

source

Compiles the one fused graph.

The base architecture’s load_model also compiles a vision encoder and allocates the MAX-side state cache. Neither applies here: the spec graph is text-only, and its pools (including the shadows) are supplied by the serving engine.

Parameters:

session (InferenceSession)

Return type:

Model