IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

apply_swiglu_activation

def apply_swiglu_activation[dtype: DType, width: SIMDLength, //, activation: def(Float32, Float32) thin -> Float32](gate: SIMD[dtype, width], up: SIMD[dtype, width]) -> SIMD[dtype, width]

Applies a gated activation lane by lane.

Parameters:

  • ​dtype (DType): Element type of the operands.
  • ​width (SIMDLength): SIMD width of the operands.
  • ​activation (def(Float32, Float32) thin -> Float32): The activation to apply.

Args:

Returns:

SIMD[dtype, width]: The activated values.

Was this page helpful?