IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

moe_create_indices_kernel

def moe_create_indices_kernel[input_type: DType, restore_type: DType, TokenExpertOrderLayoutType: TensorLayout, ExpertStartIndicesLayoutType: TensorLayout, RestoreTokenOrderLayoutType: TensorLayout, ExpertIdsLayoutType: TensorLayout, ExpertUsageStatsLayoutType: TensorLayout, TopkIdsLayoutType: TensorLayout, block_threads: Int = Int(512), _scale_alignment: UInt32 = UInt32(128), slot_layout: Bool = False, slot_stride: Int = Int(1), *, TokenExpertOrderEngine: TensorEngine, ExpertStartIndicesEngine: TensorEngine, RestoreTokenOrderEngine: TensorEngine, ExpertIdsEngine: TensorEngine, ExpertUsageStatsEngine: TensorEngine, TopkIdsEngine: TensorEngine](token_expert_order: TileTensor[.uint32, TokenExpertOrderLayoutType, MutAnyOrigin, Engine=TokenExpertOrderEngine], expert_start_indices: TileTensor[.uint32, ExpertStartIndicesLayoutType, MutAnyOrigin, Engine=ExpertStartIndicesEngine], restore_token_order: TileTensor[restore_type, RestoreTokenOrderLayoutType, MutAnyOrigin, Engine=RestoreTokenOrderEngine], expert_ids: TileTensor[.int32, ExpertIdsLayoutType, MutAnyOrigin, Engine=ExpertIdsEngine], expert_usage_stats: TileTensor[.uint32, ExpertUsageStatsLayoutType, MutAnyOrigin, Engine=ExpertUsageStatsEngine], topk_ids: TileTensor[input_type, TopkIdsLayoutType, ImmutAnyOrigin, Engine=TopkIdsEngine], scales_offset_p: Optional[Pointer[UInt32, MutAnyOrigin]], expert_offset: Int32, local_experts: Int32, slot_weights_in_p: Optional[Pointer[Float32, MutAnyOrigin]], slot_token_p: Optional[Pointer[Int32, MutAnyOrigin]], slot_weight_p: Optional[Pointer[Float32, MutAnyOrigin]], scale_slot_p: Optional[Pointer[Int32, MutAnyOrigin]], scale_rows: Int32)

Builds MoE routing indices in one CTA using a block-wide scan.

Groups tokens by their assigned expert so downstream kernels such as grouped matmul can process each expert's tokens contiguously. Every expert in [0, num_experts) gets a slot, including ones with zero tokens; expert_usage_stats holds the largest per-expert token count and num_experts.

Slot order is ascending expert id; token order within a slot follows the shared cursor. Neither is load-bearing: consumers pair expert_start_indices[g] and [g + 1] with expert_ids[g], and invert through restore_token_order.

With slot_layout, the kernel also writes what a grouped GEMM over the sorted slots reads besides the CSR (see moe_create_indices_with_slots): ids are first mapped to local groups, and each sorted slot's source row and weight, plus every scale row's source slot, are written alongside.

Was this page helpful?