IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

nvfp4_kv_dequant

NVFP4 KV-cache dequantization helpers for the SM100 sparse MLA decode kernel.

Target family: Blackwell sm_100.

An NVFP4 KV operand is a uint8 cache that packs two e2m1 elements per byte (element 0 in the low nibble) under one float8_e4m3fn scale per 16 elements. The kernel dequantizes it to FP8 in shared memory, so Q@K and P@V stay on the FP8 tcgen05.mma.kind::f8f6f4 path.

An MLA row keeps its trailing NVFP4_MLA_ROPE_DIM rope dims as raw FP8 bytes instead: the packed latent comes first, then the rope bytes, and only the latent has scales. The rope bytes are exactly what an FP8 cache holds, so the kernel copies them to shared memory without a conversion.

Each FP8 value is e2m1 * sf rounded once to e4m3. A 16-element group shares one scale and e2m1 has only 8 magnitudes, so the eight products are converted once and every element is then a byte lookup.

The e4m3_magnitude_lut / e2m1x8_to_e4m3x8 pair mirrors the helpers in Kernels/lib/msa/nvfp4_kv.mojo, which cannot be imported from max/kernels.

comptime values​

NVFP4_MLA_ROPE_DIM​

comptime NVFP4_MLA_ROPE_DIM = 64

NVFP4_SF_VECTOR_SIZE​

comptime NVFP4_SF_VECTOR_SIZE = 16

Functions​

Was this page helpful?