IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_grouped_matmul_swiglu_nvfp4

struct Struct_grouped_matmul_swiglu_nvfp4

MOGG wrapper for fused grouped NVFP4 matmul + SwiGLU + NVFP4 quant.

Fuses the MoE gate/up grouped matmul, SwiGLU activation, and per-block NVFP4 quantization into a single SM100 kernel. The caller must pre-permute the weight b and its scale tile b_scales on the N axis with sigma(2i)=i, sigma(2i+1)=D+i (where D = moe_dim, N = 2D).

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[activation: StaticString, swiglu_alpha: Float32, swiglu_limit: Float32, has_a_row_scales: Bool, max_total_m: Int, prefill_dynamic_claim: Bool](c_packed: T, c_swiglu_scales: T, a: T, b: T, a_scales: T, b_scales: T, expert_start_indices: T, expert_ids: T, a_scale_offsets: T, expert_scales: T, a_row_scales: T, c_input_scales: T, estimated_total_m: UInt32, num_active_experts: UInt32, context: DeviceContext)

Executes fused grouped NVFP4 matmul + SwiGLU + NVFP4 quant.

Computes (c_packed, c_swiglu_scales) = quantize_nvfp4(silu(C[..., even]) * C[..., odd], c_input_scales) where C = A @ B^T for multiple expert groups. Because B is sigma-permuted on N, adjacent matmul-output columns carry (gate, up) pairs that the epilogue consumes in-place.

Parameters:

  • ​activation (StaticString): Activation name, bound from the op's same-named attribute; the activation is built at compile time.
  • ​swiglu_alpha (Float32): The activation's alpha constant.
  • ​swiglu_limit (Float32): The activation's clamp limit.
  • ​has_a_row_scales (Bool): Whether a_row_scales holds per-row input scales.
  • ​max_total_m (Int): Upper bound on a's row count for the MegaFFN fusion's arrival buffer, or 0; unused by this kernel.
  • ​prefill_dynamic_claim (Bool): Whether a MegaFFN this op fuses into claims its prefill tiles from a shared counter; unused by this kernel.

Args:

  • ​c_packed (T): Packed NVFP4 output of shape (total_tokens, D // 2).
  • ​c_swiglu_scales (T): 5D FP8 SF tile in tcgen05 layout for the output.
  • ​a (T): The input tensor of shape (total_tokens, K // 2).
  • ​b (T): The sigma-permuted weight of shape (num_experts, 2D, K // 2).
  • ​a_scales (T): The A scale factors in tcgen05 5D layout.
  • ​b_scales (T): The sigma-permuted B scale factors in tcgen05 6D layout.
  • ​expert_start_indices (T): The starting token index for each expert.
  • ​expert_ids (T): The expert ID for each group.
  • ​a_scale_offsets (T): The starting scale index for each expert.
  • ​expert_scales (T): The per-expert scaling factors for the epilogue.
  • ​a_row_scales (T): Per-row input scales of shape (total_tokens,), multiplied into each row with its expert scale before the SwiGLU. Unread unless has_a_row_scales.
  • ​c_input_scales (T): Per-expert SiLU input scale (= 1/output_inv_scale).
  • ​estimated_total_m (UInt32): The estimated total number of tokens.
  • ​num_active_experts (UInt32): The number of active experts.
  • ​context (DeviceContext): The device context pointer.

Was this page helpful?