For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
grouped_matmul_block_scaled_amd
def grouped_matmul_block_scaled_amd[out_dtype: DType](c: TileTensor[out_dtype, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[.uint8, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[.uint8, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[.float8_e8m0fnu, Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[.float8_e8m0fnu, Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], row_offsets: TileTensor[.uint32, Engine=row_offsets.Engine, address_space=row_offsets.address_space, linear_idx_type=row_offsets.linear_idx_type], expert_ids: TileTensor[.int32, Engine=expert_ids.Engine, address_space=expert_ids.address_space, linear_idx_type=expert_ids.linear_idx_type], num_active_experts: Int, ctx: DeviceContext)
Launches the grouped per-expert MXFP4 block-scaled matmul kernel on AMD CDNA4.
Enqueues grouped_matmul_block_scaled_amd_kernel over a 2D grid
sized by the output M and N dimensions, emitting a trace event for
profiling.
Parameters:
- out_dtype (
DType): Element type of the output matrix.
Args:
- c (
TileTensor[out_dtype, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output matrix of shape[M, N]. - a (
TileTensor[.uint8, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): LHS activation matrix of shape[M, K//2]as packed MXFP4uint8. - b (
TileTensor[.uint8, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): RHS expert weight matrix of shape[E, N, K//2]as packed MXFP4uint8, whereEis the number of experts. - a_scales (
TileTensor[.float8_e8m0fnu, Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type]): Block scale factors foraof shape[M, K//32]asfloat8_e8m0fnu. - b_scales (
TileTensor[.float8_e8m0fnu, Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type]): Block scale factors forbof shape[E, N, K//32]asfloat8_e8m0fnu. - row_offsets (
TileTensor[.uint32, Engine=row_offsets.Engine, address_space=row_offsets.address_space, linear_idx_type=row_offsets.linear_idx_type]): Contiguous row boundaries per expert of shape[num_active_experts + 1]asuint32, where rangeispans rowsrow_offsets[i]throughrow_offsets[i + 1]. - expert_ids (
TileTensor[.int32, Engine=expert_ids.Engine, address_space=expert_ids.address_space, linear_idx_type=expert_ids.linear_idx_type]): Expert ID for each active range of shape[num_active_experts]asint32. - num_active_experts (
Int): Number of active experts to search over. - ctx (
DeviceContext): Device context used to enqueue the kernel.