For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Struct_matmul_weight_only_block_scaled_apple
struct Struct_matmul_weight_only_block_scaled_apple
Apple M5 weight-only NVFP4 (W4A16) matmul: out = a @ dequant(b)^T.
Unlike mo.matmul.dynamic.block.scaled (the NVIDIA SM100 path), the
activation a stays in bf16 (NOT dynamically quantized to FP4) and the
weight block scales are PLAIN rank-2 [N, K // 16] (NOT the SM100 rank-5
TCGEN05 interleave). The FP4 weight is dequantized to bf16 in-register at
the MMA loader seam (enqueue_apple_fp4_matmul); weights stay packed in
DRAM. The NVFP4 per-tensor weight_scale_2 scalar is applied at the graph
level by the caller (a post-matmul multiply), so it is NOT an input here.
Implemented traits
Methods
execute
static def execute[c_type: DType, //, target: StringSlice[ImmStaticOrigin]](c: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=c.static_spec], a: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=a.static_spec], b: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b.static_spec], b_scales: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b_scales.static_spec], context: DeviceContext)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!