IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

swigluoai

def swigluoai[alpha: Float32, limit: Float32](gate: Float32, up: Float32) -> Float32

Computes the clamped swigluoai activation.

With g = min(gate, limit) and u = clamp(up, -limit, limit), returns g * sigmoid(alpha * g) * (u + 1).

Parameters:

  • ​alpha (Float32): Sigmoid scale.
  • ​limit (Float32): Clamp bound.

Args:

  • ​gate (Float32): Gate projection value.
  • ​up (Float32): Up projection value.

Returns:

Float32: The activated value.

Was this page helpful?