IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_grouped_matmul_rowwise_dynamic_scaled_fp8

struct Struct_grouped_matmul_rowwise_dynamic_scaled_fp8

MOGG wrapper for grouped (ragged MoE) rowwise/per-token scaled FP8 matmul.

Serves rowwise (per-output-channel) weight scales + per-token (colwise) dynamic activation scales - the compressed-tensors FP8 layout used by e.g. RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic. On NVIDIA SM100 (B200) with a bf16 output it runs the persistent tensor-core grouped GEMM and applies the scales in its epilogue; other cases use a naive CUDA-core kernel.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute(c: T, a: T, b: T, a_scales: T, b_scales: T, expert_start_indices: T, expert_ids: T, max_num_tokens_per_expert: UInt32, num_active_experts: UInt32, context: DeviceContext)

Was this page helpful?