For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
router_gemv_partials_kernel
def router_gemv_partials_kernel[hidden_type: DType, weight_type: DType, PartialsLayoutType: TensorLayout, HiddenLayoutType: TensorLayout, GateWeightLayoutType: TensorLayout, hidden_size: Int, num_splits: Int, tile_tokens: Int, tile_experts: Int, num_threads: Int](partials: TileTensor[weight_type, PartialsLayoutType, MutAnyOrigin], hidden: TileTensor[hidden_type, HiddenLayoutType, ImmutAnyOrigin], gate_weight: TileTensor[weight_type, GateWeightLayoutType, ImmutAnyOrigin])
Writes partials[s, t, e] = hidden[t, slice s] . gate_weight[e, slice s].
Splits the hidden dimension into num_splits slices so that a decode
batch spreads over the whole GPU. Each block stages a
tile_tokens x slice block of hidden rows and a tile_experts x slice
block of gate rows in shared memory, then each thread computes one
(token, expert) dot product in weight_type.