IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MatmulFusedPartialRMSNorm

struct MatmulFusedPartialRMSNorm

Fuses GEMV (M=1 matmul) with partial RMS normalization.

Computes y = x @ W.T, then applies RMS normalization to the first N_normed columns while passing the remaining columns through unchanged.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[epsilon: Float32, weight_offset: Float32, transpose_b: Bool = True](normed_output: T, unnormed_output: T, input: T, weight: T, gamma: T, ctx: DeviceContext)

Execute fused GEMV + partial RMS norm.

Calls gemv_and_partial_norm from nn.gemv_partial_norm which computes y = x @ W.T, then partitions y into normed and unnormed outputs.

Parameters:

  • ​epsilon (Float32): Small constant added to the squared mean before the reciprocal square root for numerical stability.
  • ​weight_offset (Float32): Reserved for API consistency with other RMS norm ops; not consumed by this kernel.
  • ​transpose_b (Bool): Whether to transpose weight before the matmul (defaults to True).

Args:

  • ​normed_output (T): Output tensor holding the RMS-normalized leading columns of the matmul result.
  • ​unnormed_output (T): Output tensor holding the trailing columns of the matmul result, passed through unchanged.
  • ​input (T): Input activation tensor x of the GEMV.
  • ​weight (T): Weight matrix W of shape (N, K) (rank 2).
  • ​gamma (T): RMS normalization scale vector (rank 1).
  • ​ctx (DeviceContext): The device context used to enqueue the kernel.

Was this page helpful?