For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
quantize_dynamic_block_scaled
def quantize_dynamic_block_scaled[out_dtype: DType, scales_dtype: DType, in_dtype: DType, //, *, SF_VECTOR_SIZE: Int](output_device: TileTensor[out_dtype, Engine=output_device.Engine, linear_idx_type=output_device.linear_idx_type], scales_device: TileTensor[scales_dtype, Engine=scales_device.Engine, linear_idx_type=scales_device.linear_idx_type], input_device: TileTensor[in_dtype, Engine=input_device.Engine, linear_idx_type=input_device.linear_idx_type], tensor_sf: Float32, ctx: DeviceContext)
Dispatches dynamic block-scaled quantization to the appropriate hardware-specific kernel.
Routes to the SM100 NVFP4 and MXFP8 TMA-async path, the SM100
quantize_dynamic_scaled_fp4fp8 path, or the AMD CDNA4 MXFP4 path
based on the target device and scale-factor format.
Parameters:
- out_dtype (
DType): Element type of the quantized output tensor (inferred).uint8for NVFP4/MXFP4 orfloat8_e4m3fnfor MXFP8. - scales_dtype (
DType): Element type of the block scale-factor tensor (inferred).float8_e4m3fnfor NVFP4 orfloat8_e8m0fnufor MXFP8. - in_dtype (
DType): Element type of the input activation tensor (inferred). Must bebfloat16. - SF_VECTOR_SIZE (
Int): Number of elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8.
Args:
- output_device (
TileTensor[out_dtype, Engine=output_device.Engine, linear_idx_type=output_device.linear_idx_type]): Output quantized tensor of packed FP4uint8orfloat8_e4m3fnfor MXFP8. - scales_device (
TileTensor[scales_dtype, Engine=scales_device.Engine, linear_idx_type=scales_device.linear_idx_type]): Output block scale-factor tensor in the 5D TCGEN interleaved layout on SM100 or rank-2 layout on AMD CDNA4. - input_device (
TileTensor[in_dtype, Engine=input_device.Engine, linear_idx_type=input_device.linear_idx_type]): Input BF16 activation tensor. - tensor_sf (
Float32): Tensor-wise scale factor multiplied into each per-group scale factor. - ctx (
DeviceContext): Device context used to enqueue the kernel.