For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
fused_silu_nvfp4_kernel
def fused_silu_nvfp4_kernel[fp4_dtype: DType, scales_dtype: DType, input_dtype: DType, output_layout: TensorLayout, scales_layout: TensorLayout, input_layout: TensorLayout, offsets_layout: TensorLayout, scales_offsets_layout: TensorLayout, input_scales_layout: TensorLayout, num_threads: Int, num_sms: Int, activation: def(Float32, Float32) thin -> Float32 = comptime #kgen.func.symbol<@linalg::@swiglu_activation::@"swiglu[::SIMD[DType.float32, 1]](::SIMD[DType.float32, 1],::SIMD[DType.float32, 1])"<:!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype f32}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> apply(:!lit.generator<("value": !lit.struct<@std::@builtin::@float_literal::@FloatLiteral<:!pop.float_literal #pop.float_literal<0|1>>>, |) -> !lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype f32}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>>> @std::@simd::@SIMD::@"__init__[!pop.float_literal](::FloatLiteral[$2])"<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype f32}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}, :!pop.float_literal #pop.float_literal<0|1>>, {})>>, *, OutputEngine: TensorEngine = DefaultEngine, ScalesEngine: TensorEngine = DefaultEngine, InputEngine: TensorEngine = DefaultEngine, RowOffsetsEngine: TensorEngine = DefaultEngine, ScalesOffsetsEngine: TensorEngine = DefaultEngine, InputScalesEngine: TensorEngine = DefaultEngine](output_tensor: TileTensor[fp4_dtype, output_layout, MutUntrackedOrigin, Engine=OutputEngine], scales_tensor: TileTensor[scales_dtype, scales_layout, MutUntrackedOrigin, Engine=ScalesEngine], input_tensor: TileTensor[input_dtype, input_layout, ImmUntrackedOrigin, Engine=InputEngine], row_offsets: TileTensor[.uint32, offsets_layout, ImmUntrackedOrigin, Engine=RowOffsetsEngine], scales_offsets: TileTensor[.uint32, scales_offsets_layout, ImmUntrackedOrigin, Engine=ScalesOffsetsEngine], input_scales: TileTensor[.float32, input_scales_layout, ImmUntrackedOrigin, Engine=InputScalesEngine])
This kernel performs the gated activation for all the MLPs in the EP MoE module. We need to manually implement the kernel here is because after the EP dispatch phase, the actual number of received tokens is not known to the host. This kernel will read the row offsets to determine the actual number of received tokens in the input tensor.
Once the activation is applied, the output tensor will be quantized to
the NVFP4 format. The scales tensor will be padded and zero-filled. The
activation is the comptime activation parameter (defaults to swiglu).
Arguments: output_tensor: The output tensor to store the result. scales_tensor: The tensor to store the scales. input_tensor: The input tensor to perform the SILU operation. row_offsets: The row offsets to determine the actual number of received tokens. scales_offsets: The offsets to determine the position of the scales tiles. input_scales: Per-expert input scale factors.