For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
sigmoid_gemv_single_group_router
def sigmoid_gemv_single_group_router[hidden_type: DType, weight_type: DType, bias_type: DType, //, n_routed_experts: Int, n_experts_per_tok: Int, norm_weights: Bool, target: StringSpan[ImmStaticOrigin]](expert_indices: TileTensor[.int32, Engine=expert_indices.Engine, address_space=expert_indices.address_space, linear_idx_type=expert_indices.linear_idx_type], expert_weights: TileTensor[weight_type, Engine=expert_weights.Engine, address_space=expert_weights.address_space, linear_idx_type=expert_weights.linear_idx_type], hidden: TileTensor[hidden_type, Engine=hidden.Engine, address_space=hidden.address_space, linear_idx_type=hidden.linear_idx_type], gate_weight: TileTensor[weight_type, Engine=gate_weight.Engine, address_space=gate_weight.address_space, linear_idx_type=gate_weight.linear_idx_type], expert_bias: TileTensor[bias_type, Engine=expert_bias.Engine, address_space=expert_bias.address_space, linear_idx_type=expert_bias.linear_idx_type], routed_scaling_factor: Float32, context: DeviceContextBase)
Runs the sigmoid single-group MoE router, computing its gate GEMV.
Equivalent to single_group_router on
sigmoid(hidden.cast[weight_type]() @ gate_weight.T). The GEMV splits
the hidden dimension across blocks and writes partial dot products; the
router kernel adds them and applies the sigmoid as it loads its scores,
so no separate reduction runs.
Inputs: expert_indices: Output expert indices. Shape: [num_tokens, n_experts_per_tok]. expert_weights: Output expert weights. Shape: [num_tokens, n_experts_per_tok]. hidden: Hidden states. Shape: [num_tokens, hidden_size]. gate_weight: Router weight. Shape: [n_routed_experts, hidden_size]. expert_bias: Per-expert correction bias used for selection only. routed_scaling_factor: Scalar multiplied into every output weight. context: The device context.
Parameters:
- hidden_type (
DType): DType of the hidden states. - weight_type (
DType): DType of the gate weight, the scores and output weights. - bias_type (
DType): DType of the expert correction bias. - n_routed_experts (
Int): Total number of experts. - n_experts_per_tok (
Int): Experts selected per token. - norm_weights (
Bool): If True, normalize selected weights to sum to 1 before applying routed_scaling_factor. - target (
StringSpan[ImmStaticOrigin]): The target device to run the kernel on.