For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_grouped_matmul_block_scaled_mxfp6
struct Struct_grouped_matmul_block_scaled_mxfp6[FP6_FORMAT: Int = Int(0)]
MOGG wrapper for the grouped MXFP6 block-scaled matmul.
The FP6 sibling of matmul.grouped.block.scaled.amd, and a separate
op for the same reason the dense one is: both FP6 encodings put 24 bytes in
a lane, so lane_bytes cannot choose between them.
Preshuffled-B only. FP6's 24-byte lane fragment is plane-split (see
Shuffler.b_plane_byte_off), which the dense row-major
block_scaled_grouped_matmul_amd kernel has no path for.
Parameters
- FP6_FORMAT (
Int): 0 selects E2M3, 1 selects E3M2, matchingFP6Format.
Implemented traits
Methods
execute
static def execute(c: T, a: T, b: T, a_scales: T, b_scales: T, expert_start_indices: T, expert_ids: T, max_num_tokens_per_expert: UInt32, num_active_experts: UInt32, estimated_total_m: UInt32, decode_grid_m_cap: UInt32, decode_grid_m_rows: UInt32, context: DeviceContext)
Computes C = A @ B^T over expert groups with MXFP6 operands.
Args:
- c (
T): The output tensor of shape (total_tokens, N). - a (
T): The packed activations, (total_tokens, K * 3 // 4). - b (
T): The plane-split preshuffled weights, (num_experts, N, K * 3//4). - a_scales (
T): The A scale factors, (total_tokens, K // 32). - b_scales (
T): The B scale factors, (num_experts, N, K // 32). - expert_start_indices (
T): The starting token index for each expert. - expert_ids (
T): The expert ID for each group. - max_num_tokens_per_expert (
UInt32): The maximum token count for any expert. - num_active_experts (
UInt32): The number of active experts. - estimated_total_m (
UInt32): Estimated total received tokens for this GPU, used to pick the persistent vs direct kernel path. - decode_grid_m_cap (
UInt32): Decode-band gate selecting the direct kernel over the persistent one; 0 disables. - decode_grid_m_rows (
UInt32): Rows grid.y must cover per expert on the decode bands. - context (
DeviceContext): The device context pointer.