For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
fused_silu_mxfp8_interleaved_kernel
def fused_silu_mxfp8_interleaved_kernel[fp8_dtype: DType, scales_dtype: DType, input_dtype: DType, output_layout: TensorLayout, scales_layout: TensorLayout, input_layout: TensorLayout, offsets_layout: TensorLayout, scales_offsets_layout: TensorLayout, num_threads: Int, num_sms: Int, activation: def(Float32, Float32) thin -> Float32 = comptime #kgen.func.symbol<@linalg::@swiglu_activation::@"swiglu[::SIMD[DType.float32, 1]](::SIMD[DType.float32, 1],::SIMD[DType.float32, 1])"<:!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype f32}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> apply(:!lit.generator<("value": !lit.struct<@std::@builtin::@float_literal::@FloatLiteral<:!pop.float_literal #pop.float_literal<0|1>>>, |) -> !lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype f32}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>>> @std::@simd::@SIMD::@"__init__[!pop.float_literal](::FloatLiteral[$2])"<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype f32}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}, :!pop.float_literal #pop.float_literal<0|1>>, {})>>, *, OutputEngine: TensorEngine = DefaultEngine, ScalesEngine: TensorEngine = DefaultEngine, InputEngine: TensorEngine = DefaultEngine, RowOffsetsEngine: TensorEngine = DefaultEngine, ScalesOffsetsEngine: TensorEngine = DefaultEngine](output_tensor: TileTensor[fp8_dtype, output_layout, MutUntrackedOrigin, Engine=OutputEngine], scales_tensor: TileTensor[scales_dtype, scales_layout, MutUntrackedOrigin, Engine=ScalesEngine], input_tensor: TileTensor[input_dtype, input_layout, ImmUntrackedOrigin, Engine=InputEngine], row_offsets: TileTensor[.uint32, offsets_layout, ImmUntrackedOrigin, Engine=RowOffsetsEngine], scales_offsets: TileTensor[.uint32, scales_offsets_layout, ImmUntrackedOrigin, Engine=ScalesOffsetsEngine])
SwiGLU + MXFP8 quantization for interleaved gate/up layout.
MXFP8 counterpart of fused_silu_nvfp4_interleaved_kernel. Consumes
BF16 inputs in the [gate_0, up_0, gate_1, up_1, ...] interleaved
layout produced by permuting the MoE up-projection weight on the N
axis with sigma(2i)=i, sigma(2i+1)=H+i, applies the comptime
activation (defaults to swiglu), and quantizes the result per
32-element block to FP8-E4M3 with FP8-UE8M0
block scales.
Compared with the NVFP4 variant the differences are:
- Output is one fp8_e4m3fn byte per element (vs. 2 fp4 nibbles
per byte for NVFP4), so
output_dim == hidden_size. - Block size is
MXFP8_SF_VECTOR_SIZE = 32(vs. 16) and the per- block scale uses 4 cooperating threads vialane_group_max(vs. 2 for NVFP4). - Scale dtype is
float8_e8m0fnu(E8M0, power-of-2 only) and the scale value isblock_max / 448(FP8-E4M3 max abs), rounded to the nearest power of two by the cast to E8M0. - No per-expert
input_scales(tensor_sf) folded into the scale: E8M0 cannot represent non-power-of-2 multipliers without precision loss; MXFP8 activations use the per-block scale only. Caller'sc_input_scalesis therefore unused here and not plumbed through; the fused in-tile path matches the same contract. - 5D scale tile layout is identical to NVFP4; only the
SF_VECTOR_SIZEdivisor passed toset_scale_factorchanges.