IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

gated_residual_rms_norm_split

def gated_residual_rms_norm_split[scale: Bool](a: GatedResidualArgs, ctx: DeviceContext)

Writes the gated residual stream and its next pre-norm for fewer rows than SMs: a cluster of SPLIT CTAs per row, each CTA's warp sums gathered into rank 0 and its total sent back, summed in rms_norm's 1024-thread block tier's order so the bits match the unfused ops.

Was this page helpful?