IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MatmulFusedPartialRMSNorm

struct MatmulFusedPartialRMSNorm

Fuses GEMV (M=1 matmul) with partial RMS normalization.

Computes y = x @ W.T, then applies RMS normalization to the first N_normed columns while passing the remaining columns through unchanged.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[dtype: DType, rank: Int, target: StringSpan[ImmStaticOrigin], epsilon: Float32, weight_offset: Float32, transpose_b: Bool = True](normed_output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=normed_output.static_spec], unnormed_output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=unnormed_output.static_spec], input: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=input.static_spec], weight: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=weight.static_spec], gamma: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=gamma.static_spec], ctx: DeviceContextBase)

Execute fused GEMV + partial RMS norm.

Calls gemv_and_partial_norm from nn.gemv_partial_norm which computes y = x @ W.T, then partitions y into normed and unnormed outputs.

Parameters:

  • ​dtype (DType): Element type of all input and output tensors.
  • ​rank (Int): Tensor rank of the input and output tensors.
  • ​target (StringSpan[ImmStaticOrigin]): The target GPU device.
  • ​epsilon (Float32): Small constant added to the squared mean before the reciprocal square root for numerical stability.
  • ​weight_offset (Float32): Reserved for API consistency with other RMS norm ops; not consumed by this kernel.
  • ​transpose_b (Bool): Whether to transpose weight before the matmul (defaults to True).

Args:

Was this page helpful?