For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

quantize_mxfp6_amd

def quantize_mxfp6_amd[fmt: FP6Format, *, SF_VECTOR_SIZE: Int = Int(32), num_max_threads: Int = Int(512)](ctx: DeviceContext, output: TileTensor[Storage=output.Storage, address_space=output.address_space, linear_idx_type=output.linear_idx_type], scales: TileTensor[Storage=scales.Storage, address_space=scales.address_space, linear_idx_type=scales.linear_idx_type], input: TileTensor[Storage=input.Storage, address_space=input.address_space, linear_idx_type=input.linear_idx_type])

Quantizes bfloat16 activations to MXFP6 with E8M0 block scales.

Parameters:

• ​fmt (FP6Format): The FP6 encoding to produce (E2M3 or E3M2).
• ​SF_VECTOR_SIZE (Int): Elements each E8M0 block scale covers (defaults to 32).
• ​num_max_threads (Int): Maximum threads per block (defaults to 512).

Args:

• ​ctx (DeviceContext): Device context for kernel launch.
• ​output (TileTensor[Storage=output.Storage, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): Output tensor [num_rows, num_cols * 6 // 8] of uint8, four packed FP6 codes per three bytes.
• ​scales (TileTensor[Storage=scales.Storage, address_space=scales.address_space, linear_idx_type=scales.linear_idx_type]): Scale tensor [num_rows, num_cols // SF_VECTOR_SIZE] of float8_e8m0fnu.
• ​input (TileTensor[Storage=input.Storage, address_space=input.address_space, linear_idx_type=input.linear_idx_type]): Input tensor [num_rows, num_cols] of bfloat16.