For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
moe_create_indices_kernel
def moe_create_indices_kernel[input_type: DType, restore_type: DType, TokenExpertOrderLayoutType: TensorLayout, ExpertStartIndicesLayoutType: TensorLayout, RestoreTokenOrderLayoutType: TensorLayout, ExpertIdsLayoutType: TensorLayout, ExpertUsageStatsLayoutType: TensorLayout, TopkIdsLayoutType: TensorLayout, block_threads: Int = Int(512), _scale_alignment: UInt32 = UInt32(128), slot_layout: Bool = False, slot_stride: Int = Int(1), *, TokenExpertOrderEngine: TensorEngine, ExpertStartIndicesEngine: TensorEngine, RestoreTokenOrderEngine: TensorEngine, ExpertIdsEngine: TensorEngine, ExpertUsageStatsEngine: TensorEngine, TopkIdsEngine: TensorEngine](token_expert_order: TileTensor[.uint32, TokenExpertOrderLayoutType, MutAnyOrigin, Engine=TokenExpertOrderEngine], expert_start_indices: TileTensor[.uint32, ExpertStartIndicesLayoutType, MutAnyOrigin, Engine=ExpertStartIndicesEngine], restore_token_order: TileTensor[restore_type, RestoreTokenOrderLayoutType, MutAnyOrigin, Engine=RestoreTokenOrderEngine], expert_ids: TileTensor[.int32, ExpertIdsLayoutType, MutAnyOrigin, Engine=ExpertIdsEngine], expert_usage_stats: TileTensor[.uint32, ExpertUsageStatsLayoutType, MutAnyOrigin, Engine=ExpertUsageStatsEngine], topk_ids: TileTensor[input_type, TopkIdsLayoutType, ImmutAnyOrigin, Engine=TopkIdsEngine], scales_offset_p: Optional[Pointer[UInt32, MutAnyOrigin]], expert_offset: Int32, local_experts: Int32, slot_weights_in_p: Optional[Pointer[Float32, MutAnyOrigin]], slot_token_p: Optional[Pointer[Int32, MutAnyOrigin]], slot_weight_p: Optional[Pointer[Float32, MutAnyOrigin]], scale_slot_p: Optional[Pointer[Int32, MutAnyOrigin]], scale_rows: Int32)
Builds MoE routing indices in one CTA using a block-wide scan.
Groups tokens by their assigned expert so downstream kernels such as
grouped matmul can process each expert's tokens contiguously. Every expert
in [0, num_experts) gets a slot, including ones with zero tokens;
expert_usage_stats holds the largest per-expert token count and
num_experts.
Slot order is ascending expert id; token order within a slot follows the
shared cursor. Neither is load-bearing: consumers pair
expert_start_indices[g] and [g + 1] with expert_ids[g], and invert
through restore_token_order.
With slot_layout, the kernel also writes what a grouped GEMM over the
sorted slots reads besides the CSR (see moe_create_indices_with_slots):
ids are first mapped to local groups, and each sorted slot's source row
and weight, plus every scale row's source slot, are written alongside.