For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mxfp4_dequant_matmul_amd
def mxfp4_dequant_matmul_amd(c: TileTensor[.bfloat16, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[.bfloat16, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b_packed: TileTensor[.uint8, Engine=b_packed.Engine, address_space=b_packed.address_space, linear_idx_type=b_packed.linear_idx_type], b_scales: TileTensor[.float8_e8m0fnu, Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], ctx: DeviceContext, min_block_scale_exp: Int = Int(-1), max_block_scale_exp: Int = Int(-1))
MXFP4 matmul: native block-scaled MMA on packed B, or dequant + FP8 GEMM.
The native route only runs when the caller passes the weight's block-scale
exponent range and it proves every weight * scale is E4M3-representable
(see mxfp4_native_route_enabled), and when both weight-side views are
statically shaped and the scales dense row-major (see
mxfp4_native_views_supported); otherwise — including when the range is
unknown, the default — the dequant route runs, which is bit-exact whatever
the scales are and honours either view's strides.
Args:
- c (
TileTensor[.bfloat16, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output [M, N] in bfloat16. - a (
TileTensor[.bfloat16, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Activations [M, K] in bfloat16. - b_packed (
TileTensor[.uint8, Engine=b_packed.Engine, address_space=b_packed.address_space, linear_idx_type=b_packed.linear_idx_type]): Weights [N, K//2] in uint8 (packed MXFP4). A runtime row or column extent, or non-dense scale strides, keep the call on the dequant route. - b_scales (
TileTensor[.float8_e8m0fnu, Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type]): Weight scales [N, K//32] in float8_e8m0fnu. The native route needs this view dense row-major; any other strides are read correctly by the dequant route only. - ctx (
DeviceContext): Device context. - min_block_scale_exp (
Int): Smallest E8M0 byte amongb_scales, from the weight's repack. Defaults to unknown, which stays on the dequant route. - max_block_scale_exp (
Int): Largest E8M0 byte amongb_scales. Same default and consequence.