IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

router_gemv_partials_kernel

def router_gemv_partials_kernel[hidden_type: DType, weight_type: DType, PartialsLayoutType: TensorLayout, HiddenLayoutType: TensorLayout, GateWeightLayoutType: TensorLayout, hidden_size: Int, num_splits: Int, tile_tokens: Int, tile_experts: Int, num_threads: Int](partials: TileTensor[weight_type, PartialsLayoutType, MutAnyOrigin], hidden: TileTensor[hidden_type, HiddenLayoutType, ImmutAnyOrigin], gate_weight: TileTensor[weight_type, GateWeightLayoutType, ImmutAnyOrigin])

Writes partials[s, t, e] = hidden[t, slice s] . gate_weight[e, slice s].

Splits the hidden dimension into num_splits slices so that a decode batch spreads over the whole GPU. Each block stages a tile_tokens x slice block of hidden rows and a tile_experts x slice block of gate rows in shared memory, then each thread computes one (token, expert) dot product in weight_type.

Was this page helpful?