For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
softmax_3_pass
def softmax_3_pass[dtype: DType, InputFn1DType: def[_simd_width: Int](Int) -> SIMD[dtype, _simd_width] & RegisterPassable & ImplicitlyCopyable, //, simd_width: Int, logsoftmax: Bool = False](input_fn_1d: InputFn1DType, output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type])
Performs an unbatched softmax on an input tensor using the three-pass algorithm.
The unbatched three-pass softmax is defined as:
procedure SoftmaxUnbatched(InputInput)
maxVal = -∞
denom = 0
STEP 1: find the max value in each batch
for i = 0 to N do
maxVal = max(maxVal, Input[b, i])
end for
STEP 2: compute the exponential for each batch
for i = 0 to N do
Output[b, i] = exp(Input[b, i] - maxVal)
denom += Output[b, i]
end for
STEP 3: normalize each batch
for i = 0 to N do
Output[b, i] /= denom
end forParameters:
- dtype (
DType): The dtype of the input and output buffers (inferred). - InputFn1DType (
def[_simd_width: Int](Int) -> SIMD[dtype, _simd_width]&RegisterPassable&ImplicitlyCopyable): The type ofinput_fn_1d(inferred). - simd_width (
Int): The simd_width to use in vectorization. - logsoftmax (
Bool): Enable to apply elementwise log() to outputs after softmax.
Args:
- input_fn_1d (
InputFn1DType): The elementwise input lambda. - output (
TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): The output buffer in which to store the softmax values.