IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

mega_ffn_fused

mega_ffn_fused()​

max.nn.ep.mega_ffn_fused(config, buffers, *, tokens, ids, weights, input_scales, gate_up_weight, gate_up_b_scales, down_weight, down_b_scales, gate_up_expert_scales, down_expert_scales, c_input_scales, activation=None, layer_idx=None)

source

Runs dispatch, the routed experts’ FFN and the combine in one launch.

Each device runs one kernel, which replaces dispatch(), both grouped matmuls and combine(). Every device must run it for every MoE layer, also with no tokens, because its peers wait for its counts. The fused workspace that EP init sets up is shared by every MoE layer on a device.

Parameters:

  • config (EPConfig) – The expert-parallel configuration. EP init must have set config.fused_moe_ready.
  • buffers (EPBuffers) – The exchange’s buffers.
  • tokens (Tensor) – Each device’s BF16 tokens, of shape (tokens, hidden) per device.
  • ids (Tensor) – Each device’s int32 top-k expert ids, of shape (tokens, top_k) per device.
  • weights (Tensor) – Each device’s float32 router weights, of shape (tokens, top_k) per device.
  • input_scales (Tensor) – Each device’s inverted NVFP4 dispatch input scale. The kernel reads element 0. It is 1 for MXFP8.
  • gate_up_weight (Tensor) – The local experts’ gate/up weights, in the sigma-permuted NVFP4 or MXFP8 layout.
  • gate_up_b_scales (Tensor) – The gate/up weight scales.
  • down_weight (Tensor) – The local experts’ NVFP4 or MXFP8 down weights.
  • down_b_scales (Tensor) – The down weight scales.
  • gate_up_expert_scales (Tensor) – Each local expert’s gate/up output scale.
  • down_expert_scales (Tensor) – Each local expert’s down output scale.
  • c_input_scales (Tensor) – Each local expert’s inverted scale for the requantization between the two matmuls.
  • activation (GatedActivation | None) – The gated activation that the kernel fuses. Defaults to SiLU.
  • layer_idx (Tensor | None) – Each device’s int32 index of this MoE layer, as for dispatch(). Needed with config.eplb_count_expert_load, and ignored otherwise.

Returns:

Each token’s weighted sum of its routed experts’ BF16 outputs, mapped as tokens. The caller adds the shared expert.

Raises:

ValueError – If check_supported() rejects config, if EP init did not set up the fused MoE, or if config counts expert load without layer_idx or without expert_load.

Return type:

Tensor