IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MatmulRowInvariant

struct MatmulRowInvariant

Registers the mo.matmul.row_invariant graph op with the graph compiler.

Computes c = a @ b.T for BF16 a, b and c as mo.matmul does, except that a one-row launch stays off the GEMV and runs the multi-row dispatch instead. A model whose decode step must match its multi-row speculative verify opts in per projection. Only the SM100 dispatch honors the opt-in, so another target fails to compile, and an N = 1 output still takes the GEMV at every row count.

That removes the GEMV arm and nothing else: the SM100 heuristic it lands on still picks its kernel and its num_split_k by the runtime row count, from tables bucketed in m. Two row counts in the same bucket reduce K in the same order and so agree bit for bit; two that straddle a bucket edge do not, and that edge is a property of the (N, K) shape, not of this op. Pinning one configuration across the whole range would give the bit identity the name suggests, at the cost of the decode bucket's tuned split-K geometry above it, so it is not what this op does; a caller that needs it for a shape whose buckets differ has to pin the split itself.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute(c: T, a: T, b: T, ctx: DeviceContext)

Was this page helpful?