For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
sigmoid_gemv_group_limited_router
def sigmoid_gemv_group_limited_router[hidden_type: DType, weight_type: DType, bias_type: DType, //, n_routed_experts: Int, n_experts_per_tok: Int, n_groups: Int, topk_group: Int, norm_weights: Bool, packed: Bool = False](expert_indices: TileTensor[.int32, Engine=expert_indices.Engine, address_space=expert_indices.address_space, linear_idx_type=expert_indices.linear_idx_type], expert_weights: TileTensor[weight_type, Engine=expert_weights.Engine, address_space=expert_weights.address_space, linear_idx_type=expert_weights.linear_idx_type], hidden: TileTensor[hidden_type, Engine=hidden.Engine, address_space=hidden.address_space, linear_idx_type=hidden.linear_idx_type], gate_weight: TileTensor[weight_type, Engine=gate_weight.Engine, address_space=gate_weight.address_space, linear_idx_type=gate_weight.linear_idx_type], expert_bias: TileTensor[bias_type, Engine=expert_bias.Engine, address_space=expert_bias.address_space, linear_idx_type=expert_bias.linear_idx_type], routed_scaling_factor: Float32, context: DeviceContext, packed_p: Optional[Pointer[Scalar[weight_type], MutUntrackedOrigin]] = None)
Runs the sigmoid group-limited MoE router, computing its gate GEMV.
Equivalent to router_group_limited on
sigmoid(hidden.cast[weight_type]() @ gate_weight.T). The GEMV writes
partial dot products over hidden-dimension splits; the router adds them
and applies the sigmoid as it loads its scores, so no separate reduction
or sigmoid runs.
Inputs:
expert_indices: Output expert indices. Shape: [num_tokens, n_experts_per_tok].
expert_weights: Output expert weights. Shape: [num_tokens, n_experts_per_tok].
hidden: Hidden states. Shape: [num_tokens, hidden_size].
gate_weight: Router weight. Shape: [n_routed_experts, hidden_size].
expert_bias: Per-expert correction bias used for selection only.
routed_scaling_factor: Scalar multiplied into every output weight.
context: The device context.
packed_p: With packed, the [num_tokens, 2 * n_experts_per_tok]
output of each token's ids, as weight_type, then its weights.
Parameters:
- hidden_type (
DType): DType of the hidden states. - weight_type (
DType): DType of the gate weight, the scores and output weights. - bias_type (
DType): DType of the expert correction bias. - n_routed_experts (
Int): Total number of experts. - n_experts_per_tok (
Int): Experts selected per token. - n_groups (
Int): Number of expert groups. - topk_group (
Int): Groups kept per token. - norm_weights (
Bool): If True, normalize selected weights to sum to 1 before applying routed_scaling_factor. - packed (
Bool): Also writepacked_p.