IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

quantize_dynamic_scaled_fp4_async

def quantize_dynamic_scaled_fp4_async[input_dtype: DType, output_dtype: DType, scales_dtype: DType, //, SF_VECTOR_SIZE: Int](ctx: DeviceContext, output_tensor_tile: TileTensor[output_dtype, Engine=output_tensor_tile.Engine, linear_idx_type=output_tensor_tile.linear_idx_type], scales_tensor_tile: TileTensor[scales_dtype, Engine=scales_tensor_tile.Engine, linear_idx_type=scales_tensor_tile.linear_idx_type], input_tensor_tile: TileTensor[input_dtype, Engine=input_tensor_tile.Engine, linear_idx_type=input_tensor_tile.linear_idx_type], tensor_sf: Float32 = 1)

Launches the TMA-based NVFP4 or MXFP8 quantization kernel on SM100 hardware.

Sets up TMA tile descriptors for input, output, and scales, then enqueues the quantize_dynamic_scaled_async_kernel with warp-specialized PDL.

Parameters:

  • ​input_dtype (DType): Element type of the input activation tensor. Must be bfloat16 (inferred).
  • ​output_dtype (DType): Element type of the quantized output tensor: uint8 for NVFP4 or float8_e4m3fn for MXFP8 (inferred).
  • ​scales_dtype (DType): Element type of the block scale-factor tensor: float8_e4m3fn for NVFP4 or float8_e8m0fnu for MXFP8 (inferred).
  • ​SF_VECTOR_SIZE (Int): Number of input elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8.

Args:

Was this page helpful?