For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.architectures.eagle3_deepseekV3
Eagle3 drafts for a DeepseekV3 target.
Eagle3DeepseekV3
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3(config)
Bases: Eagle3MLADraft
Eagle3 draft paired with a DeepseekV3 target.
See Eagle3MLADraft for the implementation.
-
Parameters:
-
config (DeepseekV3Config)
has_own_lm_head
has_own_lm_head = True
Whether the head scores through its own lm_head. One whose
checkpoint ships none scores through its target’s.
Eagle3DeepseekV3DraftArchConfig
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3DraftArchConfig(max_seq_len)
Bases: ArchConfigWithBoundedMaxSeqLen
The draft architecture’s config: its sequence bound and KV leaf.
-
Parameters:
-
max_seq_len (int)
build_module()
classmethod build_module(context)
Builds the head from the draft checkpoint context names.
It embeds through its target’s embedding, so spec decoding shares that onto it, and its target’s head too when the checkpoint ships none.
-
Parameters:
-
context (ModelLoadContext)
-
Return type:
construct_kv_params()
classmethod construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype)
Returns a one-layer view of the target’s cache, for the one decoder layer the head has.
-
Parameters:
-
- huggingface_config (AutoConfig)
- pipeline_config (PipelineConfig)
- devices (list[DeviceRef])
- kv_cache_config (KVCacheConfig)
- cache_dtype (DType)
-
Return type:
initialize()
classmethod initialize(pipeline_config, model_config=None, *, max_seq_len)
Records the sequence length the pipeline resolved.
-
Parameters:
-
- pipeline_config (PipelineConfig)
- model_config (MAXModelConfig | None)
- max_seq_len (int)
-
Return type:
max_seq_len
max_seq_len: int
Eagle3DeepseekV3DraftConfig
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3DraftConfig(target_layer_ids=<factory>, *, dtype, kv_params, devices, use_subgraphs=True, data_parallel_degree=1, quantization_encoding=None, vocab_size=129280, hidden_size=7168, intermediate_size=18432, moe_intermediate_size=2048, moe_layer_freq=1, num_hidden_layers=61, num_attention_heads=128, num_key_value_heads=128, n_shared_experts=1, n_routed_experts=256, routed_scaling_factor=2.5, kv_lora_rank=512, q_lora_rank=1536, qk_rope_head_dim=64, v_head_dim=128, qk_nope_head_dim=128, topk_method='greedy', n_group=8, topk_group=4, num_experts_per_tok=8, first_k_dense_replace=3, norm_topk_prob=True, hidden_act='silu', max_position_embeddings=4096, max_seq_len, rms_norm_eps=1e-06, tie_word_embeddings=False, rope_theta=10000.0, rope_scaling=None, rope_interleave=True, scoring_func='sigmoid', attention_bias=False, attention_dropout=0.0, norm_dtype=bfloat16, gate_dtype=None, correction_bias_dtype=None, max_batch_context_length=131072, quant_config=None, attn_quant_config=None, dense_mlp_layers_without_quant=frozenset({}), ep_config=None, graph_mode='auto', return_logits=ReturnLogits.LAST_TOKEN, return_hidden_states=ReturnHiddenStates.NONE, eagle_aux_hidden_state_layer_ids=None, eplb_profile_enabled=False)
Bases: DeepseekV3Config
The target’s config, as the head runs it.
-
Parameters:
-
- target_layer_ids (list[int])
- dtype (DType)
- kv_params (KVCacheParamInterface)
- devices (list[DeviceRef])
- use_subgraphs (bool)
- data_parallel_degree (int)
- quantization_encoding (SupportedEncoding | None)
- vocab_size (int)
- hidden_size (int)
- intermediate_size (int)
- moe_intermediate_size (int)
- moe_layer_freq (int)
- num_hidden_layers (int)
- num_attention_heads (int)
- num_key_value_heads (int)
- n_shared_experts (int)
- n_routed_experts (int)
- routed_scaling_factor (float)
- kv_lora_rank (int)
- q_lora_rank (int)
- qk_rope_head_dim (int)
- v_head_dim (int)
- qk_nope_head_dim (int)
- topk_method (str)
- n_group (int)
- topk_group (int)
- num_experts_per_tok (int)
- first_k_dense_replace (int)
- norm_topk_prob (bool)
- hidden_act (str)
- max_position_embeddings (int)
- max_seq_len (int)
- rms_norm_eps (float)
- tie_word_embeddings (bool)
- rope_theta (float)
- rope_scaling (dict[str, Any] | None)
- rope_interleave (bool)
- scoring_func (str)
- attention_bias (bool)
- attention_dropout (float)
- norm_dtype (DType)
- gate_dtype (DType | None)
- correction_bias_dtype (DType | None)
- max_batch_context_length (int)
- quant_config (QuantConfig | None)
- attn_quant_config (QuantConfig | None)
- dense_mlp_layers_without_quant (frozenset[int])
- ep_config (EPConfig | None)
- graph_mode (str)
- return_logits (ReturnLogits)
- return_hidden_states (ReturnHiddenStates)
- eagle_aux_hidden_state_layer_ids (list[int] | None)
- eplb_profile_enabled (bool)
target_layer_ids
The target layers the head fuses, as MAX indices.
Eagle3MHADeepseekV3DraftArchConfig
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3MHADeepseekV3DraftArchConfig(max_seq_len)
Bases: ArchConfigWithBoundedMaxSeqLen
The MHA head’s config: its sequence bound and its own KV leaf.
-
Parameters:
-
max_seq_len (int)
build_module()
classmethod build_module(context)
Builds the MHA head from the draft checkpoint context names.
A checkpoint that doesn’t name the target layers it was trained on
gets layers spread across the target, which fc fuses as many of
as its shape says.
-
Raises:
-
ValueError – If the named layers don’t match
fc’s width. -
Parameters:
-
context (ModelLoadContext)
-
Return type:
-
tuple[Eagle3MHADraft, Eagle3MHADraftConfig]
construct_kv_params()
classmethod construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype)
Returns the head’s one-layer cache, in bfloat16.
MHA flash attention requires q/k/v and KV cache dtypes to match, and the head computes in bfloat16 whatever the target caches in.
-
Parameters:
-
- huggingface_config (AutoConfig)
- pipeline_config (PipelineConfig)
- devices (list[DeviceRef])
- kv_cache_config (KVCacheConfig)
- cache_dtype (DType)
-
Return type:
initialize()
classmethod initialize(pipeline_config, model_config=None, *, max_seq_len)
Records the sequence length the pipeline resolved.
-
Parameters:
-
- pipeline_config (PipelineConfig)
- model_config (MAXModelConfig | None)
- max_seq_len (int)
-
Return type:
max_seq_len
max_seq_len: int