For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

convert_ref_scales_to_mxfp8_format

def convert_ref_scales_to_mxfp8_format[MType: CoordLike, NType: CoordLike, KType: CoordLike, //, ref_scales_type: DType, scales_type: DType, ref_a_scales_layout: Layout, ref_b_scales_layout: Layout, a_scales_layout: Layout, b_scales_layout: Layout, a_scales_origin: MutOrigin, b_scales_origin: MutOrigin, *, REF_BLOCK_SIZE: Int, SF_VECTOR_SIZE: Int](m: MType, n: NType, k: KType, ref_a_scales: LayoutTensor[ref_scales_type, ref_a_scales_layout], ref_b_scales: LayoutTensor[ref_scales_type, ref_b_scales_layout], a_scales: LayoutTensor[scales_type, a_scales_layout, a_scales_origin], b_scales: LayoutTensor[scales_type, b_scales_layout, b_scales_origin])

Converts reference float32 block scales into the 5D MXFP8 E8M0 scale-factor layout.

Reads the per-block float32 reference scales for the A (M x K) and B (N x K) operands, converts each to float8_e8m0fnu, and writes them into the corresponding 5D scale-factor tensors expected by block-scaled matmul kernels.

Parameters:

MType (CoordLike): CoordLike type carrying the M dimension size.
NType (CoordLike): CoordLike type carrying the N dimension size.
KType (CoordLike): CoordLike type carrying the K dimension size.
ref_scales_type (DType): Element type of the reference scales (must be float32).
scales_type (DType): Element type of the output scales (must be float8_e8m0fnu).
ref_a_scales_layout (Layout): Layout of the 2D reference A scales tensor.
ref_b_scales_layout (Layout): Layout of the 2D reference B scales tensor.
a_scales_layout (Layout): Layout of the 5D output A scales tensor.
b_scales_layout (Layout): Layout of the 5D output B scales tensor.
a_scales_origin (MutOrigin): Mutability origin of the output A scales tensor.
b_scales_origin (MutOrigin): Mutability origin of the output B scales tensor.
REF_BLOCK_SIZE (Int): Block size (in elements) used by the reference scales.
SF_VECTOR_SIZE (Int): Number of elements each scale factor covers in the output layout.

Args:

m (MType): M dimension size of the operands.
n (NType): N dimension size of the operands.
k (KType): K dimension size of the operands.
ref_a_scales (LayoutTensor[ref_scales_type, ref_a_scales_layout]): 2D float32 reference scales for the A operand, indexed as [k // REF_BLOCK_SIZE, m].
ref_b_scales (LayoutTensor[ref_scales_type, ref_b_scales_layout]): 2D float32 reference scales for the B operand, indexed as [n // REF_BLOCK_SIZE, k // REF_BLOCK_SIZE].
a_scales (LayoutTensor[scales_type, a_scales_layout, a_scales_origin]): Mutable 5D output tensor receiving the converted A scales.
b_scales (LayoutTensor[scales_type, b_scales_layout, b_scales_origin]): Mutable 5D output tensor receiving the converted B scales.