IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

wmma_scale_cdna

def wmma_scale_cdna[*, a_type: DType, b_type: DType, scale_type: DType = .float8_e8m0fnu, scale16: Bool = False, a_scale_sel: Int = Int(0), b_scale_sel: Int = Int(0), a_reuse: Bool = False, b_reuse: Bool = False](mut d: SIMD[.float32], a: SIMD, b: SIMD, c: SIMD[.float32], a_scale: Scalar[DType.int64 if scale16 else DType.int32], b_scale: Scalar[DType.int64 if scale16 else DType.int32])

Computes one block-scaled CDNA5 WMMA, d = a * b * scale + c.

The instruction is v_wmma_scale, or v_wmma_scale16 with scale16, which multiplies every product of a block of the K dimension by that block's scale on top of the format's own per-element decode. A and B are expected to be the per-lane packed registers of the fragments, not one element per lane. The shape follows the accumulator width:

  • 8 f32 lanes: 16x16x128 over f8f6f4, so a_type and b_type pick each operand's format independently and 64 elements travel per lane.
  • 16 f32 lanes: 32x16x128 over f4, which is fp4 on both sides and 128 elements of A against 64 of B per lane.

The common block-scaling schemes, as the parameters they need:

Schemea_type, b_typescale_typescale16
MXFP8float8_e4m3fn, float8_e5m2float8_e8m0fnuFalse
MXFP6float6_e2m3fn, float6_e3m2fnfloat8_e8m0fnuFalse
MXFP4float4_e2m1fnfloat8_e8m0fnuFalse
NVFP4-stylefloat4_e2m1fnfloat8_e4m3fnTrue

scale16 is the ISA's v_wmma_scale16 form, which takes two 32-bit scale registers per lane rather than one. Its block size is believed to be 16 elements, against 32 for scale, but that is not confirmed here. The per-tensor fp32 scale of NVFP4 is not part of either instruction: apply it outside this call.

MXFP6 is listed for completeness but cannot be called yet: its 12-register tuple is a 12-lane vector, and a SIMD length has to be a power of two, so no A or B value can hold one. The float6 types are rejected rather than silently widened.

The ISA's third scale format, E5M3, has no DType and is not reachable.

Constraints:

  • Requires CDNA5 (gfx1250) or newer; the call fails to compile elsewhere.
  • a_type and b_type are each float8_e4m3fn, float8_e5m2 or float4_e2m1fn; scale_type is float8_e8m0fnu or float8_e4m3fn.
  • Only a float4_e2m1fn A and B pair takes float8_e4m3fn scales; every other pair takes float8_e8m0fnu.
  • The 16-lane shape is float4_e2m1fn only, because its instruction carries no format.
  • The float6 types are rejected: the 12-register tuple they ask for is a 12-lane vector, which no SIMD length can hold.
  • The widths of A and B must fill the register tuple their type asks for: 16 32-bit registers for the float8 types and 8 for float4_e2m1fn in the 8-lane shape, and 16 for A against 8 for B in the 32x16x128 shape.
  • A or B held in one of the types above must be a_type or b_type respectively; an integer container is the raw packed bits, sized by its width alone.
  • A set reuse hint asserts that this WMMA reads the same A (or B) registers as the WMMA immediately before it in the final instruction order. The compiler may reorder WMMAs, so a caller must only set a hint where that order is guaranteed.

Parameters:

  • ​a_type (DType): The element type A's registers hold.
  • ​b_type (DType): The element type B's registers hold.
  • ​scale_type (DType): The type both operands' block scales are encoded in.
  • ​scale16 (Bool): Whether to issue the two-register v_wmma_scale16 form.
  • ​a_scale_sel (Int): Which of A's scale rows to use, 0 or 1.
  • ​b_scale_sel (Int): Which of B's scale rows to use, 0 or 1.
  • ​a_reuse (Bool): Hint that this WMMA reads the same A registers as the WMMA immediately before it in the final instruction order, which lets the matrix core skip reloading them.
  • ​b_reuse (Bool): The same hint for the B registers.

Args:

Was this page helpful?