IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

quantize_dynamic_scaled_async_kernel

def quantize_dynamic_scaled_async_kernel[input_dtype: DType, input_tile_shape: Coord[*input_tile_shape.element_types], input_desc_shape: Coord[*input_desc_shape.element_types], output_dtype: DType, output_tile_shape: Coord[*output_tile_shape.element_types], output_desc_shape: Coord[*output_desc_shape.element_types], scales_dtype: DType, scales_tile_shape: Coord[*scales_tile_shape.element_types], scales_desc_shape: Coord[*scales_desc_shape.element_types], input_swizzle_mode: TensorMapSwizzle, output_swizzle_mode: TensorMapSwizzle, scales_swizzle_mode: TensorMapSwizzle, SF_VECTOR_SIZE: Int, NUM_PIPELINES_STAGES: Int](input_tma_op: TMATensorTile[input_dtype, input_tile_shape, input_desc_shape], output_tma_op: TMATensorTile[output_dtype, output_tile_shape, output_desc_shape], scales_tma_op: TMATensorTile[scales_dtype, scales_tile_shape, scales_desc_shape], tensor_sf: Float32)

GPU kernel that quantizes BF16 tiles to NVFP4 or MXFP8 using TMA async copies and warp-specialized PDL.

One warp issues TMA loads while a warpgroup computes per-group scale factors and quantizes, one row per thread, then stores results back via TMA async stores. A CTA covers 128 rows and one scale atom's SF_ATOM_K groups, loaded as NUM_PIPELINES_STAGES 64-column stages.

Parameters:

  • ​input_dtype (DType): Element type of the input activation tensor.
  • ​input_tile_shape (Coord[*element_types]): Per-dimension shape of one stage's input TMA tile.
  • ​input_desc_shape (Coord[*element_types]): Per-dimension descriptor shape of the input TMA tile.
  • ​output_dtype (DType): Element type of the quantized output tensor: uint8 for NVFP4 or float8_e4m3fn for MXFP8.
  • ​output_tile_shape (Coord[*element_types]): Per-dimension shape of the output TMA tile.
  • ​output_desc_shape (Coord[*element_types]): Per-dimension descriptor shape of the output TMA tile.
  • ​scales_dtype (DType): Element type of the block scale-factor tensor.
  • ​scales_tile_shape (Coord[*element_types]): Per-dimension shape of the scales TMA tile.
  • ​scales_desc_shape (Coord[*element_types]): Per-dimension descriptor shape of the scales TMA tile.
  • ​input_swizzle_mode (TensorMapSwizzle): TensorMapSwizzle mode for input TMA loads.
  • ​output_swizzle_mode (TensorMapSwizzle): TensorMapSwizzle mode for output TMA stores.
  • ​scales_swizzle_mode (TensorMapSwizzle): TensorMapSwizzle mode for scales TMA stores.
  • ​SF_VECTOR_SIZE (Int): Number of elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8.
  • ​NUM_PIPELINES_STAGES (Int): Number of 64-column input stages a CTA loads: 1 for NVFP4 or 2 for MXFP8.

Args:

Was this page helpful?