For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_grouped_matmul_swiglu_nvfp4
struct Struct_grouped_matmul_swiglu_nvfp4
MOGG wrapper for fused grouped NVFP4 matmul + SwiGLU + NVFP4 quant.
Fuses the MoE gate/up grouped matmul, SwiGLU activation, and per-block
NVFP4 quantization into a single SM100 kernel. The caller must pre-permute
the weight b and its scale tile b_scales on the N axis with
sigma(2i)=i, sigma(2i+1)=D+i (where D = moe_dim, N = 2D).
Implemented traits
Methods
execute
static def execute[activation: StaticString, swiglu_alpha: Float32, swiglu_limit: Float32, has_a_row_scales: Bool, max_total_m: Int, prefill_dynamic_claim: Bool](c_packed: T, c_swiglu_scales: T, a: T, b: T, a_scales: T, b_scales: T, expert_start_indices: T, expert_ids: T, a_scale_offsets: T, expert_scales: T, a_row_scales: T, c_input_scales: T, estimated_total_m: UInt32, num_active_experts: UInt32, context: DeviceContext)
Executes fused grouped NVFP4 matmul + SwiGLU + NVFP4 quant.
Computes (c_packed, c_swiglu_scales) = quantize_nvfp4(silu(C[..., even]) * C[..., odd], c_input_scales)
where C = A @ B^T for multiple expert groups. Because B is
sigma-permuted on N, adjacent matmul-output columns carry
(gate, up) pairs that the epilogue consumes in-place.
Parameters:
- activation (
StaticString): Activation name, bound from the op's same-named attribute; the activation is built at compile time. - swiglu_alpha (
Float32): The activation's alpha constant. - swiglu_limit (
Float32): The activation's clamp limit. - has_a_row_scales (
Bool): Whethera_row_scalesholds per-row input scales. - max_total_m (
Int): Upper bound ona's row count for the MegaFFN fusion's arrival buffer, or 0; unused by this kernel. - prefill_dynamic_claim (
Bool): Whether a MegaFFN this op fuses into claims its prefill tiles from a shared counter; unused by this kernel.
Args:
- c_packed (
T): Packed NVFP4 output of shape (total_tokens, D // 2). - c_swiglu_scales (
T): 5D FP8 SF tile in tcgen05 layout for the output. - a (
T): The input tensor of shape (total_tokens, K // 2). - b (
T): The sigma-permuted weight of shape (num_experts, 2D, K // 2). - a_scales (
T): The A scale factors in tcgen05 5D layout. - b_scales (
T): The sigma-permuted B scale factors in tcgen05 6D layout. - expert_start_indices (
T): The starting token index for each expert. - expert_ids (
T): The expert ID for each group. - a_scale_offsets (
T): The starting scale index for each expert. - expert_scales (
T): The per-expert scaling factors for the epilogue. - a_row_scales (
T): Per-row input scales of shape (total_tokens,), multiplied into each row with its expert scale before the SwiGLU. Unread unlesshas_a_row_scales. - c_input_scales (
T): Per-expert SiLU input scale (= 1/output_inv_scale). - estimated_total_m (
UInt32): The estimated total number of tokens. - num_active_experts (
UInt32): The number of active experts. - context (
DeviceContext): The device context pointer.