For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

fused_silu_mxfp6_kernel

def fused_silu_mxfp6_kernel[scales_dtype: DType, input_dtype: DType, output_layout: TensorLayout, scales_layout: TensorLayout, input_layout: TensorLayout, offsets_layout: TensorLayout, num_threads: Int, num_sms: Int, *, fp6_format: FP6Format, fuse_a_scale_preshuffle: Bool = False, clamp_activation: Bool = False](output_tensor: TileTensor[DType.uint8, output_layout, MutUntrackedOrigin], scales_tensor: TileTensor[scales_dtype, scales_layout, MutUntrackedOrigin], input_tensor: TileTensor[input_dtype, input_layout, ImmUntrackedOrigin], row_offsets: TileTensor[DType.uint32, offsets_layout, ImmUntrackedOrigin], alpha: Float32 = 0, limit: Float32 = 0)

SwiGLU + MXFP6 quantization for the EP MoE down-projection input.

MXFP6 counterpart of fused_silu_mx_kernel, which serves MXFP4 and MXFP8 from one body by switching on elements-per-byte. FP6 cannot join it: four codes per three bytes is a ratio of 4/3, which that integer truncates to 1 and so cannot distinguish from FP8.

The structural departure is that a thread owns a whole 32-element MX block rather than 8 elements. At FP4 and FP8 a block is split across 2 or 4 cooperating threads, each holding a whole number of bytes; at FP6 an 8-element slice is 6 bytes, which is neither a legal SIMD width nor a legal alignment, and a 4-element code group would straddle two threads' bytes. A whole block is exactly 24 bytes and one scale, so the amax reduction stays in registers -- no lane_group_max -- and the store is three aligned 8-byte writes.