IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

ReduceRMSNormFusedResidualAdd

struct ReduceRMSNormFusedResidualAdd

Registers the mo.composite.rms_norm_fused_residual_add graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[dtype: DType, rank: Int, target: StringSpan[ImmStaticOrigin], epsilon1: Float32, epsilon2: Float32, weight_offset1: Float32, weight_offset2: Float32, multiply_before_cast: Bool = True](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], residual_output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=residual_output.static_spec], input: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=input.static_spec], residual_input: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=residual_input.static_spec], gamma1: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=gamma1.static_spec], gamma2: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=gamma2.static_spec], ctx: DeviceContextBase)

Executes the mo.composite.rms_norm_fused_residual_add graph op.

Parameters:

  • ​dtype (DType): Element type of the input and output tensors.
  • ​rank (Int): Tensor rank of the input and output tensors.
  • ​target (StringSpan[ImmStaticOrigin]): Compilation target string.
  • ​epsilon1 (Float32): Stability constant for the first normalization.
  • ​epsilon2 (Float32): Stability constant for the second normalization.
  • ​weight_offset1 (Float32): Scalar offset for the first weight.
  • ​weight_offset2 (Float32): Scalar offset for the second weight.
  • ​multiply_before_cast (Bool): See the graph op signature.

Args:

Raises:

Error: If the operation parameters are invalid.

Was this page helpful?