For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
GatedResidualRMSNorm
struct GatedResidualRMSNorm
Registers the gated_residual_rms_norm graph op.
Implemented traits
Methods
execute
static def execute[scale_branch: Bool](stream_out: T, normed_out: T, branch: T, residual: T, post_weight: T, gate: T, norm_weight: T, post_eps: Float32, norm_eps: Float32, temperature: Float32, gate_eps: Float32, ctx: DeviceContext)
Writes the gated residual stream and its next pre-norm.
Parameters:
- scale_branch (
Bool): Whether the branch is scaled by1 + post_weightbefore its norm; otherwisepost_weightis not read.
Args:
- stream_out (
T): The new residual stream[rows, 8192]. - normed_out (
T): The stream after the next pre-norm[rows, 8192]. - branch (
T): The branch output[rows, 8192]. - residual (
T): The incoming residual stream[rows, 8192]. - post_weight (
T): The branch's input scale weight[8192]. - gate (
T): The residual gate's logits[8192]. - norm_weight (
T): The next pre-norm's weight[8192]. - post_eps (
Float32): The branch norm's epsilon. - norm_eps (
Float32): The next pre-norm's epsilon. - temperature (
Float32): The residual gate's temperature. - gate_eps (
Float32): The floor under the residual coefficient's root. - ctx (
DeviceContext): Device context.
Raises:
Error: If the launch fails.