For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Struct_grouped_matmul_rowwise_dynamic_scaled_fp8
struct Struct_grouped_matmul_rowwise_dynamic_scaled_fp8
MOGG wrapper for grouped (ragged MoE) rowwise/per-token scaled FP8 matmul.
Serves rowwise (per-output-channel) weight scales + per-token (colwise)
dynamic activation scales - the compressed-tensors FP8 layout used by
e.g. RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic. Targets
NVIDIA SM100 (B200) with a correctness-first naive grouped kernel.
Implemented traits
Methods
execute
static def execute[c_type: DType, a_type: DType, b_type: DType, a_scales_type: DType, b_scales_type: DType, //, target: StringSlice[ImmStaticOrigin]](c: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=c.static_spec], a: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=a.static_spec], b: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b.static_spec], a_scales: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=a_scales.static_spec], b_scales: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b_scales.static_spec], expert_start_indices: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=expert_start_indices.static_spec], expert_ids: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=expert_ids.static_spec], max_num_tokens_per_expert: UInt32, num_active_experts: UInt32, context: DeviceContext)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!