For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
wmma_scale_cdna
def wmma_scale_cdna[*, a_type: DType, b_type: DType, scale_type: DType = .float8_e8m0fnu, scale16: Bool = False, a_scale_sel: Int = Int(0), b_scale_sel: Int = Int(0), a_reuse: Bool = False, b_reuse: Bool = False](mut d: SIMD[.float32], a: SIMD, b: SIMD, c: SIMD[.float32], a_scale: Scalar[DType.int64 if scale16 else DType.int32], b_scale: Scalar[DType.int64 if scale16 else DType.int32])
Computes one block-scaled CDNA5 WMMA, d = a * b * scale + c.
The instruction is v_wmma_scale, or v_wmma_scale16 with scale16,
which multiplies every product of a block of the K dimension by that
block's scale on top of the format's own per-element decode. A and B are
expected to be the per-lane packed registers of the fragments, not one
element per lane. The shape follows the accumulator width:
- 8 f32 lanes:
16x16x128overf8f6f4, soa_typeandb_typepick each operand's format independently and 64 elements travel per lane. - 16 f32 lanes:
32x16x128overf4, which is fp4 on both sides and 128 elements of A against 64 of B per lane.
The common block-scaling schemes, as the parameters they need:
| Scheme | a_type, b_type | scale_type | scale16 |
|---|---|---|---|
| MXFP8 | float8_e4m3fn, float8_e5m2 | float8_e8m0fnu | False |
| MXFP6 | float6_e2m3fn, float6_e3m2fn | float8_e8m0fnu | False |
| MXFP4 | float4_e2m1fn | float8_e8m0fnu | False |
| NVFP4-style | float4_e2m1fn | float8_e4m3fn | True |
scale16 is the ISA's v_wmma_scale16 form, which takes two 32-bit scale
registers per lane rather than one. Its block size is believed to be 16
elements, against 32 for scale, but that is not confirmed here. The
per-tensor fp32 scale of NVFP4 is not part of either instruction: apply it
outside this call.
MXFP6 is listed for completeness but cannot be called yet: its 12-register tuple is a 12-lane vector, and a SIMD length has to be a power of two, so no A or B value can hold one. The float6 types are rejected rather than silently widened.
The ISA's third scale format, E5M3, has no DType and is not reachable.
Constraints:
- Requires CDNA5 (gfx1250) or newer; the call fails to compile elsewhere.
a_typeandb_typeare eachfloat8_e4m3fn,float8_e5m2orfloat4_e2m1fn;scale_typeisfloat8_e8m0fnuorfloat8_e4m3fn.- Only a
float4_e2m1fnA and B pair takesfloat8_e4m3fnscales; every other pair takesfloat8_e8m0fnu. - The 16-lane shape is
float4_e2m1fnonly, because its instruction carries no format. - The float6 types are rejected: the 12-register tuple they ask for is a 12-lane vector, which no SIMD length can hold.
- The widths of A and B must fill the register tuple their type asks
for: 16 32-bit registers for the float8 types and 8 for
float4_e2m1fnin the 8-lane shape, and 16 for A against 8 for B in the 32x16x128 shape. - A or B held in one of the types above must be
a_typeorb_typerespectively; an integer container is the raw packed bits, sized by its width alone. - A set reuse hint asserts that this WMMA reads the same A (or B) registers as the WMMA immediately before it in the final instruction order. The compiler may reorder WMMAs, so a caller must only set a hint where that order is guaranteed.
Parameters:
- a_type (
DType): The element type A's registers hold. - b_type (
DType): The element type B's registers hold. - scale_type (
DType): The type both operands' block scales are encoded in. - scale16 (
Bool): Whether to issue the two-registerv_wmma_scale16form. - a_scale_sel (
Int): Which of A's scale rows to use, 0 or 1. - b_scale_sel (
Int): Which of B's scale rows to use, 0 or 1. - a_reuse (
Bool): Hint that this WMMA reads the same A registers as the WMMA immediately before it in the final instruction order, which lets the matrix core skip reloading them. - b_reuse (
Bool): The same hint for the B registers.
Args:
- d (
SIMD[.float32]): The destination fragment, overwritten with the result. - a (
SIMD): The A fragment: the format's packed registers in any container that fills the 32-bit register tuple it asks for, such as 16int32lanes of fp8 or 64_uint4lanes of fp4, ora_typeelements. - b (
SIMD): The B fragment's packed registers, the same way. - c (
SIMD[.float32]): The accumulator fragment. - a_scale (
Scalar[DType.int64 if scale16 else DType.int32]): A's block scales, 32-bit per lane, or 64-bit withscale16. - b_scale (
Scalar[DType.int64 if scale16 else DType.int32]): B's block scales, the same way.