IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

single_group_router_eplb_kernel

def single_group_router_eplb_kernel[scores_type: DType, bias_type: DType, ExpertIndicesLayoutType: TensorLayout, ExpertIndicesLogLayoutType: TensorLayout, ExpertWeightsLayoutType: TensorLayout, ExpertScoresLayoutType: TensorLayout, ExpertBiasLayoutType: TensorLayout, LogcntLayoutType: TensorLayout, Log2phyLayoutType: TensorLayout, LayerIdxLayoutType: TensorLayout, n_routed_experts: Int, n_experts_per_tok: Int, norm_weights: Bool, num_threads: Int, num_log: Int, max_replicas: Int, hash_decorrelate: Bool, ScoresInputFnType: def[width: Int](Coord[**?]) -> SIMD[scores_type, width] & RegisterPassable & ImplicitlyCopyable](expert_indices: TileTensor[.int32, ExpertIndicesLayoutType, MutAnyOrigin], expert_indices_log: TileTensor[.int32, ExpertIndicesLogLayoutType, MutAnyOrigin], expert_weights: TileTensor[scores_type, ExpertWeightsLayoutType, MutAnyOrigin], expert_bias: TileTensor[bias_type, ExpertBiasLayoutType, ImmutAnyOrigin], logcnt: TileTensor[.int32, LogcntLayoutType, ImmutAnyOrigin], log2phy: TileTensor[.int32, Log2phyLayoutType, ImmutAnyOrigin], layer_idx: TileTensor[.int32, LayerIdxLayoutType, ImmutAnyOrigin], routed_scaling_factor: Float32, scores_input_fn: ScoresInputFnType)

Single-group MoE router fused with EPLB log->phy remap.

Selects with _block_top_k, then at the K writers performs the EPLB lookup using a per-block SMEM cache of the current layer's logcnt/log2phy slice. Scores are read only through scores_input_fn, a closure of type ScoresInputFnType.

Backend specialization:

  • NVIDIA: cp.async issues the table fetch up front; sort hides the latency.
  • AMD/Apple: plain ld_global into registers up front, ds_write later.

Was this page helpful?