For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
HeadRMSNormSigmoidGate
struct HeadRMSNormSigmoidGate
Registers the norm.head_rms_norm_sigmoid_gate graph op.
Implemented traits
Methods
execute
static def execute[n_heads: Int](output: T, x: T, qkvg: T, epsilon: Float32, ctx: DeviceContext)
Writes rms_norm(x) * sigmoid(gate) per 64-channel head, with the gate the last n_heads * 64 columns of qkvg.
Parameters:
- n_heads (
Int): Heads inx(this device's).
Args:
- output (
T):[tokens, n_heads * 64]. - x (
T): The attention output[tokens, n_heads * 64]. - qkvg (
T): The[q | k | v | gate]projection. - epsilon (
Float32): The norm's epsilon. - ctx (
DeviceContext): Device context.
Raises:
Error: If the launch fails.