IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

swiglu_quantize_mx_amd

def swiglu_quantize_mx_amd[out_dtype: DType = .float8_e4m3fn, scales_dtype: DType = .float8_e8m0fnu, in_dtype: DType = .bfloat16, //, OutputEngine: TensorEngine, ScalesEngine: TensorEngine, GateUpEngine: TensorEngine, *, num_max_threads: Int = Int(512), activation: def(Float32, Float32) thin -> Float32](ctx: DeviceContext, output_tile: TileTensor[out_dtype, Engine=OutputEngine, linear_idx_type=output_tile.linear_idx_type], scales_tile: TileTensor[scales_dtype, Engine=ScalesEngine, linear_idx_type=scales_tile.linear_idx_type], gate_up_tile: TileTensor[in_dtype, Engine=GateUpEngine, linear_idx_type=gate_up_tile.linear_idx_type])

Quantizes the gated activation of a BF16 [gate | up] tensor to MXFP4 or MXFP8 on AMD CDNA4 (MI355X).

One kernel computes what applying activation to each (gate, up) pair, rounding to bfloat16, and then quantize_mx_amd would, without writing the activation out. Produces the same 2D E8M0 block scales as the ungated kernel: [M, N // 32] for a [M, 2 * N] input, whose launch geometry is computed on N.

Parameters:

  • ​out_dtype (DType): Element type of the quantized output (inferred). uint8 for packed MXFP4 or float8_e4m3fn for MXFP8.
  • ​scales_dtype (DType): Element type of the block scale-factor tensor (inferred). Must be float8_e8m0fnu.
  • ​in_dtype (DType): Element type of the input (inferred). Must be bfloat16.
  • ​OutputEngine (TensorEngine): Engine policy of the output_tile tensor.
  • ​ScalesEngine (TensorEngine): Engine policy of the scales_tile tensor.
  • ​GateUpEngine (TensorEngine): Engine policy of the gate_up_tile tensor.
  • ​num_max_threads (Int): Maximum number of threads per block for the launch grid (defaults to 512). Must be at least 4, and is rounded down to a whole number of scale-block lane groups so every block stays inside one group.
  • ​activation (def(Float32, Float32) thin -> Float32): The gated activation, applied to each (gate, up) pair.

Args:

Raises:

Error: If output_tile is not [M, N] (MXFP8) or [M, N // 2] (MXFP4) or scales_tile is not [M, N // 32] for a [M, 2 * N] gate_up_tile.

Was this page helpful?