For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
single_group_router_kernel
def single_group_router_kernel[scores_type: DType, ExpertIndicesLayoutType: TensorLayout, ExpertWeightsLayoutType: TensorLayout, ExpertScoresLayoutType: TensorLayout, n_routed_experts: Int, n_experts_per_tok: Int, norm_weights: Bool, num_threads: Int, ScoresInputFnType: def[width: Int](Coord[**?]) -> SIMD[scores_type, width] & RegisterPassable & ImplicitlyCopyable, BiasInputFnType: def[width: Int](Coord[**?]) -> SIMD[scores_type, width] & RegisterPassable & ImplicitlyCopyable, *, ExpertIndicesEngine: TensorEngine, ExpertWeightsEngine: TensorEngine, packed: Bool = False](expert_indices: TileTensor[.int32, ExpertIndicesLayoutType, MutAnyOrigin, Engine=ExpertIndicesEngine], expert_weights: TileTensor[scores_type, ExpertWeightsLayoutType, MutAnyOrigin, Engine=ExpertWeightsEngine], num_tokens: Int32, routed_scaling_factor: Float32, packed_p: Optional[Pointer[Scalar[scores_type], MutAnyOrigin]], scores_input_fn: ScoresInputFnType, bias_input_fn: BiasInputFnType)
Single-group MoE router kernel. One warp per token.
Fuses: corrected = scores + bias β top-k selection β weight = the
selected scores β optional normalize β scale. Each lane holds a few
adjacent experts of every warp-wide span, and each of the
n_experts_per_tok rounds takes the warp's largest corrected score,
ties to the lower expert.
Scores and the selection bias are read only through scores_input_fn
and bias_input_fn, closures that load the value at a
(token, expert) index, so the bias may differ per token.
With packed, each token's ids, as scores_type, then its weights
are also written to its [2 * n_experts_per_tok] row of packed_p.