For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
softmax_inline
def softmax_inline[dtype: DType, simd_width: Int, rank: Int, InputFnType: def[_simd_width: Int](Coord[**?]) -> SIMD[dtype, _simd_width] & RegisterPassable & ImplicitlyCopyable, logsoftmax: Bool = False, has_prologue_fusion: Bool = True](input_fn: InputFnType, shape: Coord[*shape.element_types], output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], axis: Int, context: Optional[DeviceContext[target]] = DeviceContext[_].NoneValue)
Dispatches softmax (or logsoftmax) to the CPU or GPU target.
Selects the appropriate CPU or GPU implementation based on target and
traces the operation. Exits early when the tensor is empty.
Parameters:
- dtype (
DType): The dtype of the input and output buffers. - simd_width (
Int): The simd_width to use in vectorization. - rank (
Int): The rank of the input and output tensors. - InputFnType (
def[_simd_width: Int](Coord[**?]) -> SIMD[dtype, _simd_width]&RegisterPassable&ImplicitlyCopyable): The type ofinput_fn(inferred). - logsoftmax (
Bool): Enable to apply elementwise log() to outputs after softmax. - has_prologue_fusion (
Bool): Whether the input lambda supports prologue fusion.
Args:
- input_fn (
InputFnType): The elementwise input lambda. On GPU it runs on the device, so it must copy what it captures. - shape (
Coord[*shape.element_types]): The shape of the output tensor. - output (
TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): The output buffer in which to store the softmax values. - axis (
Int): The axis along which to compute the softmax. - context (
Optional[DeviceContext[target]]): Optional device context for GPU execution.