For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
MatmulRowInvariant
struct MatmulRowInvariant
Registers the mo.matmul.row_invariant graph op with the graph compiler.
Computes c = a @ b.T for BF16 a, b and c as mo.matmul does,
except that a one-row launch stays off the GEMV and runs the multi-row
dispatch instead. A model whose decode step must match its multi-row
speculative verify opts in per projection. Only the SM100 dispatch honors
the opt-in, so another target fails to compile, and an N = 1 output
still takes the GEMV at every row count.
That removes the GEMV arm and nothing else: the SM100 heuristic it lands on
still picks its kernel and its num_split_k by the runtime row count, from
tables bucketed in m. Two row counts in the same bucket reduce K in the
same order and so agree bit for bit; two that straddle a bucket edge do
not, and that edge is a property of the (N, K) shape, not of this op.
Pinning one configuration across the whole range would give the bit
identity the name suggests, at the cost of the decode bucket's tuned
split-K geometry above it, so it is not what this op does; a caller that
needs it for a shape whose buckets differ has to pin the split itself.
Implemented traitsโ
Methodsโ
executeโ
static def execute(c: T, a: T, b: T, ctx: DeviceContext)