For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
dequant_mx
def dequant_mx(ctx: DeviceContext, output: TileTensor[Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], input: TileTensor[Engine=input.Engine, address_space=input.address_space, linear_idx_type=input.linear_idx_type], scales: TileTensor[Engine=scales.Engine, address_space=scales.address_space, linear_idx_type=scales.linear_idx_type], num_rows: Int, num_cols: Int)
Dequantize MXFP4 or MXFP8 weights to bfloat16 or float32.
The format follows the dtype of input: uint8 is packed MXFP4,
float8_e4m3fn is MXFP8.
Args:
- ctx (
DeviceContext): Device context for kernel launch. - output (
TileTensor[Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): Output tensor[num_rows, num_cols]of bfloat16 or float32. - input (
TileTensor[Engine=input.Engine, address_space=input.address_space, linear_idx_type=input.linear_idx_type]): Packed MXFP4 weights[num_rows, num_cols // 2]of uint8, or MXFP8 weights[num_rows, num_cols]of float8_e4m3fn. - scales (
TileTensor[Engine=scales.Engine, address_space=scales.address_space, linear_idx_type=scales.linear_idx_type]): Block scales offloat8_e8m0fnu: rank 2[num_rows, num_cols // 32], or the rank-5 interleaved layout[ceildiv(num_rows, 128), ceildiv(num_cols, 128), 32, 4, 4]. - num_rows (
Int): Number of rows (N dimension for weights). - num_cols (
Int): Number of columns (K dimension, unpacked). Must be a multiple of 32.
Raises:
If num_cols is not a multiple of 32, or if the shapes of output,
input and scales do not agree with num_rows and num_cols. The
kernels read scales on trust, so a short scale tensor would otherwise
be an out-of-bounds read.