IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

sigmoid_gemv_single_group_router

def sigmoid_gemv_single_group_router[hidden_type: DType, weight_type: DType, bias_type: DType, //, n_routed_experts: Int, n_experts_per_tok: Int, norm_weights: Bool, target: StringSpan[ImmStaticOrigin]](expert_indices: TileTensor[.int32, Engine=expert_indices.Engine, address_space=expert_indices.address_space, linear_idx_type=expert_indices.linear_idx_type], expert_weights: TileTensor[weight_type, Engine=expert_weights.Engine, address_space=expert_weights.address_space, linear_idx_type=expert_weights.linear_idx_type], hidden: TileTensor[hidden_type, Engine=hidden.Engine, address_space=hidden.address_space, linear_idx_type=hidden.linear_idx_type], gate_weight: TileTensor[weight_type, Engine=gate_weight.Engine, address_space=gate_weight.address_space, linear_idx_type=gate_weight.linear_idx_type], expert_bias: TileTensor[bias_type, Engine=expert_bias.Engine, address_space=expert_bias.address_space, linear_idx_type=expert_bias.linear_idx_type], routed_scaling_factor: Float32, context: DeviceContextBase)

Runs the sigmoid single-group MoE router, computing its gate GEMV.

Equivalent to single_group_router on sigmoid(hidden.cast[weight_type]() @ gate_weight.T). The GEMV splits the hidden dimension across blocks and writes partial dot products; the router kernel adds them and applies the sigmoid as it loads its scores, so no separate reduction runs.

Inputs: expert_indices: Output expert indices. Shape: [num_tokens, n_experts_per_tok]. expert_weights: Output expert weights. Shape: [num_tokens, n_experts_per_tok]. hidden: Hidden states. Shape: [num_tokens, hidden_size]. gate_weight: Router weight. Shape: [n_routed_experts, hidden_size]. expert_bias: Per-expert correction bias used for selection only. routed_scaling_factor: Scalar multiplied into every output weight. context: The device context.

Parameters:

  • ​hidden_type (DType): DType of the hidden states.
  • ​weight_type (DType): DType of the gate weight, the scores and output weights.
  • ​bias_type (DType): DType of the expert correction bias.
  • ​n_routed_experts (Int): Total number of experts.
  • ​n_experts_per_tok (Int): Experts selected per token.
  • ​norm_weights (Bool): If True, normalize selected weights to sum to 1 before applying routed_scaling_factor.
  • ​target (StringSpan[ImmStaticOrigin]): The target device to run the kernel on.

Was this page helpful?