For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
quantize_dynamic_scaled_async_kernel
def quantize_dynamic_scaled_async_kernel[input_dtype: DType, input_tile_shape: Coord[*input_tile_shape.element_types], input_desc_shape: Coord[*input_desc_shape.element_types], output_dtype: DType, output_tile_shape: Coord[*output_tile_shape.element_types], output_desc_shape: Coord[*output_desc_shape.element_types], scales_dtype: DType, scales_tile_shape: Coord[*scales_tile_shape.element_types], scales_desc_shape: Coord[*scales_desc_shape.element_types], input_swizzle_mode: TensorMapSwizzle, output_swizzle_mode: TensorMapSwizzle, scales_swizzle_mode: TensorMapSwizzle, SF_VECTOR_SIZE: Int, NUM_PIPELINES_STAGES: Int](input_tma_op: TMATensorTile[input_dtype, input_tile_shape, input_desc_shape], output_tma_op: TMATensorTile[output_dtype, output_tile_shape, output_desc_shape], scales_tma_op: TMATensorTile[scales_dtype, scales_tile_shape, scales_desc_shape], tensor_sf: Float32)
GPU kernel that quantizes BF16 tiles to NVFP4 or MXFP8 using TMA async copies and warp-specialized PDL.
One warp issues TMA loads while a warpgroup computes per-group scale
factors and quantizes, one row per thread, then stores results back via
TMA async stores. A CTA covers 128 rows and one scale atom's
SF_ATOM_K groups, loaded as NUM_PIPELINES_STAGES 64-column stages.
Parameters:
- input_dtype (
DType): Element type of the input activation tensor. - input_tile_shape (
Coord[*element_types]): Per-dimension shape of one stage's input TMA tile. - input_desc_shape (
Coord[*element_types]): Per-dimension descriptor shape of the input TMA tile. - output_dtype (
DType): Element type of the quantized output tensor:uint8for NVFP4 orfloat8_e4m3fnfor MXFP8. - output_tile_shape (
Coord[*element_types]): Per-dimension shape of the output TMA tile. - output_desc_shape (
Coord[*element_types]): Per-dimension descriptor shape of the output TMA tile. - scales_dtype (
DType): Element type of the block scale-factor tensor. - scales_tile_shape (
Coord[*element_types]): Per-dimension shape of the scales TMA tile. - scales_desc_shape (
Coord[*element_types]): Per-dimension descriptor shape of the scales TMA tile. - input_swizzle_mode (
TensorMapSwizzle):TensorMapSwizzlemode for input TMA loads. - output_swizzle_mode (
TensorMapSwizzle):TensorMapSwizzlemode for output TMA stores. - scales_swizzle_mode (
TensorMapSwizzle):TensorMapSwizzlemode for scales TMA stores. - SF_VECTOR_SIZE (
Int): Number of elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8. - NUM_PIPELINES_STAGES (
Int): Number of 64-column input stages a CTA loads: 1 for NVFP4 or 2 for MXFP8.
Args:
- input_tma_op (
TMATensorTile[input_dtype, input_tile_shape, input_desc_shape]): TMA tile descriptor for async input loads. - output_tma_op (
TMATensorTile[output_dtype, output_tile_shape, output_desc_shape]): TMA tile descriptor for async output stores. - scales_tma_op (
TMATensorTile[scales_dtype, scales_tile_shape, scales_desc_shape]): TMA tile descriptor for async scale-factor stores. - tensor_sf (
Float32): Tensor-wise scale factor applied to the NVFP4 quantization; unused for MXFP8.