IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

router_gate_gemm

Large-M router-gate GEMM for SM100 (B200): bf16 activation, fp32 weight.

Target family: Blackwell sm_100. Shape of record: MiniMax-M3 prefill, [16384, 6144] bf16 x [128, 6144] fp32^T -> [16384, 128] fp32.

The baseline widens the activation to fp32 in a standalone pass and then runs an fp32 matmul, which the SM100 dispatch executes with TF32 tensor cores. That moves the [M, K] activation through HBM three times (read bf16, write fp32, read fp32). A bf16 value is exact in TF32, so only the weight needs TF32 rounding, and the router weight is tiny (N * K). This route therefore splits each fp32 weight into two bf16 values whose sum is that weight truncated to TF32's 10-bit mantissa, runs one bf16 tensor-core GEMM over the interleaved [2N, K] weight, and adds each output pair in the GEMM epilogue. The activation is read once, as bf16.

Functions​

Was this page helpful?