For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

activations

The module contains implementations of activation functions.

Functions

• ​elu: Compute the Elu Op using the equation $z if z >= 0 else alpha*(e^z -1)$.
• ​gelu: Compute the exact GELU activation using the equation $0.5 * x * (1 + erf(x / sqrt(2)))$.
• ​gelu_quick: Compute the quick (sigmoid) approximation of the GELU activation: $x * sigmoid(1.702 * x)$.
• ​gelu_tanh: Compute the tanh approximation of the GELU activation: $0.5 * x * (1 + tanh(0.7978845608028654 * (x + 0.044715 * x^3)))$.
• ​leaky_relu: Compute the Leaky ReLU using the equation $max(x, 0) + negative_slope * min(x, 0)$.
• ​relu: Compute the Relu Op using the equation $max(x, 0)$.
• ​relu_n1: Compute the Relu N1 Op using the equation $max(min(x,1),-1)$.
• ​sigmoid: Compute the sigmoid activation using the equation $1 / (1 + e^{-x})$.
• ​sign: Compute the sign (0, 1) of the input value.
• ​silu: Compute the SiLU (Swish) activation using the equation $x * sigmoid(x)$.