IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

head_norm

def head_norm(x: Piece, eps: Float32) -> Piece

The lane's piece of rms_norm over its 8-lane head row.

Adds as rowwise's warp tier does at 64 channels: lane l of 32 sums channels 2l, 2l + 1, then strides 16, 8, 4, 2, 1. This lane holds that warp's lanes 4j .. 4j + 3: three strides are shuffles, two are local.

Returns:

Piece

Was this page helpful?