For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
conv_utils
Utilities for direct convolution kernels on CPU.
Provides convolution shape description, L2 tiling heuristics, micro-kernel shape selection, and work partitioning helpers used by the direct and im2col convolution implementations.
comptime values
ConvEpilogueFn
comptime ConvEpilogueFn = def(Coord[**?], Int) -> None & RegisterPassable & ImplicitlyCopyable
Fused epilogue applied over a whole f_size-long run at coords.
Distinct from ConvSimdEpilogueFn, which receives the value; this one receives
only the coordinate and the run length. Rank-N in the output's own space; the
Coord carries its rank, so there is no rank parameter.
ConvSimdComputeFn
comptime ConvSimdComputeFn = def[dtype: DType, width: SIMDLength, alignment: Int = Int(1)](Coord[**?], SIMD[dtype, width]) -> SIMD[dtype, width] & RegisterPassable & ImplicitlyCopyable
Fused convolution compute epilogue: (coords, value) -> value.
The coordinates are rank-N in the output's own space (NHWC or NDHWC). The kernel stores the returned value into its output, so the closure never captures the output and can travel next to it.
ConvSimdEpilogueFn
comptime ConvSimdEpilogueFn = def[dtype: DType, width: SIMDLength, alignment: Int = Int(1)](Coord[**?], SIMD[dtype, width]) -> None & RegisterPassable & ImplicitlyCopyable
Fused elementwise store epilogue for convolution: (coord, value) -> None.
The coordinate is rank-N in the output's own space (NHWC / NDHWC), unlike
linalg.utils.ElementwiseEpilogueFn, which is always rank 2. The Coord
carries its own rank. The closure stores the value itself, so it must not
capture the output tensor passed alongside it; use ConvSimdComputeFn there.
no_conv_simd_compute_fn
comptime no_conv_simd_compute_fn = _as_conv_simd_compute_fn[inflated$def[comptime[dtype: DType, width: SIMDLength, alignment: Int = Int(1)] #kgen.func.symbol<@nn::@conv::@conv_utils::@"_identity_conv_simd_compute_body[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>]](.__init__())
Stands in for a ConvSimdComputeFn argument on code paths that apply no compute epilogue.
no_conv_simd_epilogue_fn
comptime no_conv_simd_epilogue_fn = _as_conv_simd_epilogue_fn[inflated$def[comptime[dtype: DType, width: SIMDLength, alignment: Int = Int(1)] #kgen.func.symbol<@nn::@conv::@conv_utils::@"_no_conv_simd_epilogue_body[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>]](.__init__())
Stands in for a ConvSimdEpilogueFn argument on code paths that store the output directly.
Structs
-
ConvAlgorithm: Tags the convolution algorithm selected for a given operation. -
ConvInfoStatic: Holds statically known convolution attributes (padding, stride, dilation, groups). -
ConvPartition: Work range for a partition. -
ConvShape: A shape struct describing the convolution dimensions. -
NoConvEpilogue: Stands in for aConvEpilogueFnargument on code paths that apply no epilogue.
Functions
-
align_down_residual: Returns the remainder after aligning down value to alignment. -
append_shape: Append input shape by insertinglast2ndandlastat the end. -
extend_shape: Extend input shape by insertingfirstandlastat both ends. -
get_conv_num_partitions: Partition the workload in (batch, C, F, HOWO) dimensions. HOWO is the combination of HO and WO dimensions. The actual number of tasks are the product of return num_partitions. -
get_conv_num_tasks: Returns the number of tasks to partition the convolution into. -
get_conv_shape: Builds aConvShapefrom the output, input, and filter tile tensors. -
get_conv_tile_shape: Compute the (c, f) tile shape in L2. -
get_conv_tile_size: Returns the convolution L2 cache tile size in elements for the target. -
get_direct_conv_micro_kernel_height: Returns the micro-kernel height for direct convolution on the target CPU. -
get_direct_conv_micro_kernel_width: Returns the micro-kernel width for direct convolution on the target CPU. -
get_micro_kernel_shape: Selects the(height, width)micro-kernel tile shape for the target. -
get_partition: Computes theConvPartitionwork range for a given task id.