For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
swiglu
def swiglu[limit: Float32](gate: Float32, up: Float32) -> Float32
Computes the SwiGLU activation silu(gate) * up.
A non-zero limit first clamps the inputs to min(gate, limit) and
clamp(up, -limit, limit).
Parameters:
- โlimit (
Float32): Clamp bound, or 0 for no clamp.
Args:
Returns:
Float32: The activated value.