IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

sigmoid_gemv_group_limited_router

def sigmoid_gemv_group_limited_router[hidden_type: DType, weight_type: DType, bias_type: DType, //, n_routed_experts: Int, n_experts_per_tok: Int, n_groups: Int, topk_group: Int, norm_weights: Bool, packed: Bool = False](expert_indices: TileTensor[.int32, Engine=expert_indices.Engine, address_space=expert_indices.address_space, linear_idx_type=expert_indices.linear_idx_type], expert_weights: TileTensor[weight_type, Engine=expert_weights.Engine, address_space=expert_weights.address_space, linear_idx_type=expert_weights.linear_idx_type], hidden: TileTensor[hidden_type, Engine=hidden.Engine, address_space=hidden.address_space, linear_idx_type=hidden.linear_idx_type], gate_weight: TileTensor[weight_type, Engine=gate_weight.Engine, address_space=gate_weight.address_space, linear_idx_type=gate_weight.linear_idx_type], expert_bias: TileTensor[bias_type, Engine=expert_bias.Engine, address_space=expert_bias.address_space, linear_idx_type=expert_bias.linear_idx_type], routed_scaling_factor: Float32, context: DeviceContext, packed_p: Optional[Pointer[Scalar[weight_type], MutUntrackedOrigin]] = None)

Runs the sigmoid group-limited MoE router, computing its gate GEMV.

Equivalent to router_group_limited on sigmoid(hidden.cast[weight_type]() @ gate_weight.T). The GEMV writes partial dot products over hidden-dimension splits; the router adds them and applies the sigmoid as it loads its scores, so no separate reduction or sigmoid runs.

Inputs: expert_indices: Output expert indices. Shape: [num_tokens, n_experts_per_tok]. expert_weights: Output expert weights. Shape: [num_tokens, n_experts_per_tok]. hidden: Hidden states. Shape: [num_tokens, hidden_size]. gate_weight: Router weight. Shape: [n_routed_experts, hidden_size]. expert_bias: Per-expert correction bias used for selection only. routed_scaling_factor: Scalar multiplied into every output weight. context: The device context. packed_p: With packed, the [num_tokens, 2 * n_experts_per_tok] output of each token's ids, as weight_type, then its weights.

Parameters:

  • ​hidden_type (DType): DType of the hidden states.
  • ​weight_type (DType): DType of the gate weight, the scores and output weights.
  • ​bias_type (DType): DType of the expert correction bias.
  • ​n_routed_experts (Int): Total number of experts.
  • ​n_experts_per_tok (Int): Experts selected per token.
  • ​n_groups (Int): Number of expert groups.
  • ​topk_group (Int): Groups kept per token.
  • ​norm_weights (Bool): If True, normalize selected weights to sum to 1 before applying routed_scaling_factor.
  • ​packed (Bool): Also write packed_p.

Was this page helpful?