IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

ReduceRMSNormFusedResidualAdd

struct ReduceRMSNormFusedResidualAdd

Registers the mo.composite.rms_norm_fused_residual_add graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[epsilon1: Float32, epsilon2: Float32, weight_offset1: Float32, weight_offset2: Float32, multiply_before_cast: Bool = True](output: T, residual_output: T, input: T, residual_input: T, gamma1: T, gamma2: T, ctx: DeviceContext)

Executes the mo.composite.rms_norm_fused_residual_add graph op.

Parameters:

  • ​epsilon1 (Float32): Stability constant for the first normalization.
  • ​epsilon2 (Float32): Stability constant for the second normalization.
  • ​weight_offset1 (Float32): Scalar offset for the first weight.
  • ​weight_offset2 (Float32): Scalar offset for the second weight.
  • ​multiply_before_cast (Bool): See the graph op signature.

Args:

  • ​output (T): Output tensor receiving the result.
  • ​residual_output (T): See the graph op signature.
  • ​input (T): Input tensor to reduce.
  • ​residual_input (T): Residual tensor added to the normalized input.
  • ​gamma1 (T): Scale weights for the first normalization.
  • ​gamma2 (T): Scale weights for the second normalization.
  • ​ctx (DeviceContext): Device context used to enqueue the kernel.

Raises:

Error: If the operation parameters are invalid.

Was this page helpful?