IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

swiglu_quantize_dynamic_block_scaled

def swiglu_quantize_dynamic_block_scaled[out_dtype: DType, scales_dtype: DType, in_dtype: DType, //, *, SF_VECTOR_SIZE: Int, activation: def(Float32, Float32) thin -> Float32](output_device: TileTensor[out_dtype, Engine=output_device.Engine, linear_idx_type=output_device.linear_idx_type], scales_device: TileTensor[scales_dtype, Engine=scales_device.Engine, linear_idx_type=scales_device.linear_idx_type], gate_up_device: TileTensor[in_dtype, Engine=gate_up_device.Engine, linear_idx_type=gate_up_device.linear_idx_type], tensor_sf: Float32, ctx: DeviceContext)

Quantizes the gated activation of a [gate | up] input to NVFP4, MXFP4, or MXFP8 with block scales.

One kernel computes what activation(gate, up) rounded to in_dtype and then quantize_dynamic_block_scaled would, without writing the activation out. SM100 and SM120 run every supported format; AMD CDNA4 (MI355X) runs MXFP4 and MXFP8, with rank-2 row-major E8M0 scales.

Parameters:

  • ​out_dtype (DType): Element type of the quantized output (inferred). uint8 for NVFP4/MXFP4 or float8_e4m3fn for MXFP8.
  • ​scales_dtype (DType): Element type of the block scales (inferred). float8_e4m3fn for NVFP4 or float8_e8m0fnu for MXFP4/MXFP8.
  • ​in_dtype (DType): Element type of the input (inferred). Must be bfloat16.
  • ​SF_VECTOR_SIZE (Int): Elements per block scale: 16 for NVFP4, 32 for MXFP4 and MXFP8.
  • ​activation (def(Float32, Float32) thin -> Float32): The gated activation, applied to each (gate, up) pair.

Args:

Raises:

Error: If output_device is not [M, N] for MXFP8 or [M, N // 2] for packed FP4.

Was this page helpful?