IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

enqueue_matmul2d_fp4

def enqueue_matmul2d_fp4[c_type: DType = .float32, num_sg_m: Int = Int(2), num_sg_n: Int = Int(2), tm: Int = Int(2), tn: Int = Int(2)](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[.bfloat16, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], packed: TileTensor[.uint8, Engine=packed.Engine, address_space=packed.address_space, linear_idx_type=packed.linear_idx_type], scales: TileTensor[.float8_e4m3fn, Engine=scales.Engine, address_space=scales.address_space, linear_idx_type=scales.linear_idx_type], ctx: DeviceContext)

Enqueue the matmul2d W4A16 GEMM: out = a @ dequant(packed, scales)^T.

a is the bf16 activation (M, K), packed the FP4 weight (N, K//2) (lo-nibble first), scales the FP8-E4M3 block scales (N, ceil(K/16)). C is (M, N). Interior-only fast path: caller must ensure N is a multiple of the threadgroup N-tile (32*num_sg_n*tn) and K % 16 == 0 (the W4A16 production shapes are aligned). M is rounded up and the store guards partial tiles.

The 16x32x16 MMA is the native matmul2d_mma_regc_bt_native tiling (two _mma_apple_transposable calls); pure Mojo over _mma_apple.

Parameters:

  • ​c_type (DType): Output element type; one of fp16, bf16, fp32 (defaults to float32). Accumulation is fp32.
  • ​num_sg_m (Int): Simdgroup rows per threadgroup (defaults to 2).
  • ​num_sg_n (Int): Simdgroup cols per threadgroup (defaults to 2).
  • ​tm (Int): Output 16x32 tiles per simdgroup along M (defaults to 2).
  • ​tn (Int): Output 16x32 tiles per simdgroup along N (defaults to 2). tm*tn == 4 is the M5 register-cliff optimum.

Args:

Raises:

If the attached GPU is older than Apple M5 (compute_capability >= 5).

Was this page helpful?