IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

swiglu_activation

Gated activations for the fused SwiGLU epilogues.

Kernels that fuse a gated activation take it as a comptime thin function, so each activation compiles to its own specialization. get_swiglu_activation builds one from a name and its f32 scalar constants.

comptime values​

get_swiglu_activation​

comptime get_swiglu_activation[name: StringSpan[ImmStaticOrigin], alpha: Float32 = 0, limit: Float32 = 0] = _named_swiglu_activation[name, alpha, limit]

The activation for a name, for a kernel's activation parameter. Unknown names fail to compile.

Parameters​

  • ​name (StringSpan[ImmStaticOrigin]): Activation name, as the Python graph API spells it.
  • ​alpha (Float32): Constant for activations that take one; ignored otherwise.
  • ​limit (Float32): Constant for activations that take one; ignored otherwise.

swiglu_activation_type​

comptime swiglu_activation_type = def(Float32, Float32) thin -> Float32

Gated activation (gate, up) -> out.

Thin so it can ride as a comptime parameter; any constants it uses must be comptime values.

Functions​

Was this page helpful?