For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.architectures.mtp_qwen3_5
The MTP head a Qwen3.5 checkpoint ships beside its trunk.
MTPQwen3_5DraftArchConfig
class max.pipelines.architectures.mtp_qwen3_5.MTPQwen3_5DraftArchConfig(*, hidden_size, num_attention_heads, num_key_value_heads, num_hidden_layers, rope_theta, rope_scaling_params, max_seq_len, intermediate_size, interleaved_rope_weights, vocab_size, dtype, model_quantization_encoding, quantization_config, kv_params, return_logits=ReturnLogits.LAST_TOKEN, norm_method='rms_norm', norm_dtype=None, attention_bias=False, rms_norm_eps=None, tie_word_embeddings=False, stacked_mlp=False, stacked_qkv=False, attention_multiplier, embedding_multiplier, residual_multiplier, devices, clip_qkv, quant_config=None, longrope_scaling_params=None, logits_scaling=1.0, return_hidden_states=ReturnHiddenStates.NONE, target_layer_ids=None, use_subgraphs=True, data_parallel_degree=1, sliding_window=None, quantization_encoding=None, layer_types=<factory>, full_attention_interval=4, linear_key_head_dim=128, linear_value_head_dim=128, linear_num_key_heads=16, linear_num_value_heads=48, linear_conv_kernel_dim=4, partial_rotary_factor=0.25, attn_output_gate=True, mamba_ssm_dtype=float32, state_pool_dtype=None, vision_config=None, mrope_without_encoder=False, image_token_id=None, video_token_id=None, vision_start_token_id=None, mrope_section=None, hf_quantization_config=None, quant_scheme=None, declared_dtype=None)
Bases: Qwen3_5Config
The head’s config: its target’s, with a KV leaf for its one layer.
-
Parameters:
-
- hidden_size (int)
- num_attention_heads (int)
- num_key_value_heads (int)
- num_hidden_layers (int)
- rope_theta (float)
- rope_scaling_params (Llama3RopeScalingParams | None)
- max_seq_len (int)
- intermediate_size (int)
- interleaved_rope_weights (bool)
- vocab_size (int)
- dtype (DType)
- model_quantization_encoding (QuantizationEncoding | None)
- quantization_config (QuantizationConfig | None)
- kv_params (KVCacheParamInterface)
- return_logits (ReturnLogits)
- norm_method (Literal['rms_norm', 'layer_norm'])
- norm_dtype (DType | None)
- attention_bias (bool)
- rms_norm_eps (float | None)
- tie_word_embeddings (bool)
- stacked_mlp (bool)
- stacked_qkv (bool)
- attention_multiplier (float)
- embedding_multiplier (float)
- residual_multiplier (float)
- devices (list[DeviceRef])
- clip_qkv (float | None)
- quant_config (QuantConfig | None)
- longrope_scaling_params (LongRoPEScalingParams | None)
- logits_scaling (float)
- return_hidden_states (ReturnHiddenStates)
- target_layer_ids (list[int] | None)
- use_subgraphs (bool)
- data_parallel_degree (int)
- sliding_window (int | None)
- quantization_encoding (SupportedEncoding | None)
- layer_types (list[str])
- full_attention_interval (int)
- linear_key_head_dim (int)
- linear_value_head_dim (int)
- linear_num_key_heads (int)
- linear_num_value_heads (int)
- linear_conv_kernel_dim (int)
- partial_rotary_factor (float)
- attn_output_gate (bool)
- mamba_ssm_dtype (DType)
- state_pool_dtype (DType | None)
- vision_config (VisionConfig | None)
- mrope_without_encoder (bool)
- image_token_id (int | None)
- video_token_id (int | None)
- vision_start_token_id (int | None)
- mrope_section (list[int] | None)
- hf_quantization_config (dict[str, Any] | None)
- quant_scheme (Qwen3_5QuantScheme | None)
- declared_dtype (DType | None)
build_module()
classmethod build_module(context)
Builds the MTP head from the target’s checkpoint, which
context names.
The head’s config is its target’s, finalized against the head’s weights. It embeds and projects through the target’s embedding and head, so spec decoding shares those onto it.
-
Raises:
-
ValueError – If the checkpoint ships no MTP head, or one with tensors the head doesn’t read.
-
Parameters:
-
context (ModelLoadContext)
-
Return type:
-
tuple[Qwen3_5MTP, Qwen3_5Config]
construct_kv_params()
classmethod construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype, *, allow_kv_head_replication=False)
Returns a one-layer view of the target’s attention cache, for the one decoder layer the head has.
-
Parameters:
-
- huggingface_config (AutoConfig)
- pipeline_config (PipelineConfig)
- devices (list[DeviceRef])
- kv_cache_config (KVCacheConfig)
- cache_dtype (DType)
- allow_kv_head_replication (bool)
-
Return type: