For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
MoEQuantized
MoEQuantized
class max.nn.MoEQuantized(devices, hidden_dim, num_experts, num_experts_per_token, moe_dim, num_logical_experts=None, gate_cls=<class 'max.nn.moe.moe.MoEGate'>, mlp_cls=<class 'max.nn.linear.MLP'>, shared_mlp_cls=None, has_shared_experts=False, shared_experts_dim=0, ep_size=1, dtype=bfloat16, activation=None, interleaved_gate_up=False, pre_expert_norm_cls=None, ep_batch_manager=None, quant_config=None, shared_experts_dtype=None, shared_experts_quant_config=None, is_sharding=False)
Bases: MoE
Mixture of Experts with FP8 or NVFP4 quantization.
-
Parameters:
-
- devices (list[DeviceRef])
- hidden_dim (int)
- num_experts (int)
- num_experts_per_token (int)
- moe_dim (int)
- num_logical_experts (int | None)
- gate_cls (Callable[..., MoEGate])
- mlp_cls (Callable[..., MLP])
- shared_mlp_cls (Callable[..., MLP] | None)
- has_shared_experts (bool)
- shared_experts_dim (int)
- ep_size (int)
- dtype (DType)
- activation (GatedActivation | None)
- interleaved_gate_up (bool)
- pre_expert_norm_cls (Callable[[], Module] | None)
- ep_batch_manager (EPBatchManager | None)
- quant_config (QuantConfig | None)
- shared_experts_dtype (DType | None)
- shared_experts_quant_config (QuantConfig | None)
- is_sharding (bool)
down_proj_scales
property down_proj_scales: TensorValue
Returns stacked down-projection weight scales.
gate_up_proj_scales
property gate_up_proj_scales: TensorValue
Returns stacked gate/up weight scales for grouped matmul.