IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

head_norm_gate

def head_norm_gate[n_heads: Int, poison: Int = Int(0)](a: GateArgs, ctx: DeviceContext)

Writes rms_norm(x) * sigmoid(gate) per head.

Rows are 16-byte aligned. poison plants a fault for the tests.

Was this page helpful?