For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_grouped_matmul_rowwise_dynamic_scaled_fp8
struct Struct_grouped_matmul_rowwise_dynamic_scaled_fp8
MOGG wrapper for grouped (ragged MoE) rowwise/per-token scaled FP8 matmul.
Serves rowwise (per-output-channel) weight scales + per-token (colwise)
dynamic activation scales - the compressed-tensors FP8 layout used by
e.g. RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic. On NVIDIA
SM100 (B200) with a bf16 output it runs the persistent tensor-core grouped
GEMM and applies the scales in its epilogue; other cases use a naive
CUDA-core kernel.
Implemented traitsโ
Methodsโ
executeโ
static def execute(c: T, a: T, b: T, a_scales: T, b_scales: T, expert_start_indices: T, expert_ids: T, max_num_tokens_per_expert: UInt32, num_active_experts: UInt32, context: DeviceContext)