For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
router_gate_gemm
Large-M router-gate GEMM for SM100 (B200): bf16 activation, fp32 weight.
Target family: Blackwell sm_100. Shape of record: MiniMax-M3 prefill,
[16384, 6144] bf16 x [128, 6144] fp32^T -> [16384, 128] fp32.
The baseline widens the activation to fp32 in a standalone pass and then runs
an fp32 matmul, which the SM100 dispatch executes with TF32 tensor cores. That
moves the [M, K] activation through HBM three times (read bf16, write fp32,
read fp32). A bf16 value is exact in TF32, so only the weight needs TF32
rounding, and the router weight is tiny (N * K). This route therefore
splits each fp32 weight into two bf16 values whose sum is that weight
truncated to TF32's 10-bit mantissa, runs one bf16 tensor-core GEMM over the
interleaved [2N, K] weight, and adds each output pair in the GEMM epilogue.
The activation is read once, as bf16.
Functions
-
router_gate_tf32_gemm: Computesc = a @ b^Tfor bf16aand fp32bwith TF32-class numerics.