For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
RMSNormResidualAdd
struct RMSNormResidualAdd
Fused single-norm residual-add + RMSNorm.
Computes intermediate = input + residual_input and
output = rms_norm(intermediate, gamma, epsilon, weight_offset) in a
single launch, returning both output and intermediate. This is the
canonical transformer/mamba pre-norm boundary rms_norm(residual + out)
where the pre-add value is carried forward as the next block's residual.
GPUs run the rowwise kernel; the CPU reuses the state_space
fused-residual kernel.
Implemented traitsโ
Methodsโ
executeโ
static def execute[epsilon: Float32, weight_offset: Float32, multiply_before_cast: Bool = True](output: T, residual_output: T, input: T, residual_input: T, gamma: T, ctx: DeviceContext)