For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
situ
def situ[alpha: Float32, limit: Float32](gate: Float32, up: Float32) -> Float32
Computes the SiTU gated activation.
With beta = alpha and linear_beta = limit, returns
beta * tanh(gate / beta) * sigmoid(gate) * u, where u is
linear_beta * tanh(up / linear_beta) when limit != 0 and up
otherwise. Unlike the stock activations this uses IEEE fp32 division
rather than recip, like the graph's elementwise SiTU chain: the
quantize consumes the value's bfloat16 rounding, so it must track the
chain within its transcendental ulps, which recip would exceed.
Parameters:
- βalpha (
Float32): The gate's tanh scale (beta); non-zero, since the gate is divided by it. - βlimit (
Float32): The up half's soft saturation (linear_beta); 0 passesupthrough unchanged.
Args:
Returns:
Float32: The activated value.