IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

router_gemv_partials_warp_kernel

def router_gemv_partials_warp_kernel[hidden_type: DType, weight_type: DType, PartialsLayoutType: TensorLayout, HiddenLayoutType: TensorLayout, GateWeightLayoutType: TensorLayout, n_routed_experts: Int, hidden_size: Int, num_splits: Int, num_threads: Int](partials: TileTensor[weight_type, PartialsLayoutType, MutUntrackedOrigin], hidden: TileTensor[hidden_type, HiddenLayoutType, ImmUntrackedOrigin], gate_weight: TileTensor[weight_type, GateWeightLayoutType, ImmUntrackedOrigin])

Writes the partials of router_gemv_partials_kernel for a few tokens.

Each warp owns one (expert, split) pair and keeps that slice of the gate row in registers, so the gate weights are read once from global memory whatever the token count. A 16-token tile would leave most threads of router_gemv_partials_kernel idle on a decode step.

Was this page helpful?