IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

nvfp4_dequant

NVFP4 weight dequantization.

Converts packed NVFP4 weights (uint8, two E2M1 values per byte, low nibble first) with float8_e4m3fn block scales and a tensor-wide float32 scale into bfloat16 or float32:

out[r, c] = (e2m1(nibble) * |block_scale[r, c // 16]|) * tensor_sf

tensor_sf is the decode scale, i.e. the checkpoint's weight_scale_2, the same value the NVFP4 matmul applies to its output. It is not the inverted factor the quantize op takes. The multiplication order matches the host-side reference (max.pipelines.weights.fp4_quantization.dequantize_nvfp4): the E2M1 value times the block scale is exact in float32, so only the final multiply rounds, and float32 output is bit-identical to the reference.

Two block-scale layouts are accepted, selected by the rank of scales:

  • Rank 2, [N, K // 16]: the checkpoint layout.
  • Rank 5, [ceildiv(N, 128), ceildiv(K, 64), 32, 4, 4]: the SM100 TCGEN05 interleaved layout that quantize_dynamic_block_scaled emits on SM100/SM120.

E4M3 NaN scale codes are not special-cased: they propagate as NaN.

The kernel is an elementwise over [N, K] with the scale-block width as the SIMD width, so one thread dequantizes one 16-element scale block: one 8-byte packed load, one scale load, and one contiguous store of 16 outputs. Occupancy does not depend on K, and consecutive threads touch consecutive bytes of every tensor.

GPU only: the graph compiler does not support FP8 tensors on a CPU device, so there is no way to reach a CPU path from a graph.

Functions​

Was this page helpful?