IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

GatedResidualRMSNorm

struct GatedResidualRMSNorm

Registers the gated_residual_rms_norm graph op.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[scale_branch: Bool](stream_out: T, normed_out: T, branch: T, residual: T, post_weight: T, gate: T, norm_weight: T, post_eps: Float32, norm_eps: Float32, temperature: Float32, gate_eps: Float32, ctx: DeviceContext)

Writes the gated residual stream and its next pre-norm.

Parameters:

  • ​scale_branch (Bool): Whether the branch is scaled by 1 + post_weight before its norm; otherwise post_weight is not read.

Args:

  • ​stream_out (T): The new residual stream [rows, 8192].
  • ​normed_out (T): The stream after the next pre-norm [rows, 8192].
  • ​branch (T): The branch output [rows, 8192].
  • ​residual (T): The incoming residual stream [rows, 8192].
  • ​post_weight (T): The branch's input scale weight [8192].
  • ​gate (T): The residual gate's logits [8192].
  • ​norm_weight (T): The next pre-norm's weight [8192].
  • ​post_eps (Float32): The branch norm's epsilon.
  • ​norm_eps (Float32): The next pre-norm's epsilon.
  • ​temperature (Float32): The residual gate's temperature.
  • ​gate_eps (Float32): The floor under the residual coefficient's root.
  • ​ctx (DeviceContext): Device context.

Raises:

Error: If the launch fails.

Was this page helpful?