For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
dispatch_fused_im2col_conv2d_apple
def dispatch_fused_im2col_conv2d_apple[input_type: DType, filter_type: DType, output_type: DType, EpilogueFnType: ConvSimdEpilogueFn = inflated$def[comptime[dtype: DType, width: SIMDLength, alignment: Int = Int(1)] #kgen.func.symbol<@nn::@conv::@conv_utils::@"_no_conv_simd_epilogue_body[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>], ComputeFnType: ConvSimdComputeFn = inflated$def[comptime[dtype: DType, width: SIMDLength, alignment: Int = Int(1)] #kgen.func.symbol<@nn::@conv::@conv_utils::@"_identity_conv_simd_compute_body[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>], //, filter_is_fcrs: Bool = False, has_epilogue_fn: Bool = False, has_compute_fn: Bool = False](input: TileTensor[input_type, Engine=input.Engine, address_space=input.address_space, linear_idx_type=input.linear_idx_type], filter: TileTensor[filter_type, Engine=filter.Engine, address_space=filter.address_space, linear_idx_type=filter.linear_idx_type], output: TileTensor[output_type, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], stride: IndexList[Int(2)], dilation: IndexList[Int(2)], symmetric_padding: IndexList[Int(2)], num_groups: Int, ctx: DeviceContext, epilogue_fn: EpilogueFnType = no_conv_simd_epilogue_fn, compute_fn: ComputeFnType = no_conv_simd_compute_fn) -> Bool
Apple M5 fused online-im2col conv2d (no [M, K] scratch materialised).
Apple silicon (compute_capability >= 5). Mirrors the MI355 conv pattern:
the filter is transposed to the (C_out, K=R*S*C_in) NK matrix once (the
same kernel the materialised path uses), then enqueue_apple_conv2d runs the
simdgroup-tiled GEMM with the A operand gathered from the NHWC input on the
fly -- so the im2col matrix never touches global memory. This wins across
both compute- and memory-bound regimes, removing the materialised path's
memory-bound loss (and the conv_gpu Apple memory-bound naive guard).
Returns True if handled; False to fall back. Self-gates: bf16, groups=1,
dilation=1, kernel > 1x1, K=RSC_in >= 16. Unlike the materialised
dispatcher, there is NO N=C_out >= 16 gate: the fused path has no [M, K]
scratch round-trip, so small C_out (down to 1) takes the MMA path and beats
the naive thread-per-pixel conv (which is also broken on Metal for the
C_out=3 VAE->RGB shape). The MMA handles tiny N via the existing edge-tile
mask (b_valid_cols zero-fill in the B load + acol < n in the epilogue),
all comptime-fixed by SG_N=32 and independent of the runtime N.
Parameters:
- input_type (
DType): ElementDTypeof the input activation tensor (inferred). - filter_type (
DType): ElementDTypeof the filter tensor (inferred). - output_type (
DType): ElementDTypeof the output tensor (inferred). - EpilogueFnType (
ConvSimdEpilogueFn): Type ofepilogue_fn(inferred). - ComputeFnType (
ConvSimdComputeFn): Type ofcompute_fn(inferred). - filter_is_fcrs (
Bool): True if the filter is laid out as[F, C, R, S]; False for[R, S, C, F](defaults toFalse). - has_epilogue_fn (
Bool): True whenepilogue_fnstores the output instead of the GEMM (defaults toFalse). - has_compute_fn (
Bool): True when the GEMM storescompute_fnof each output value (defaults toFalse).
Args:
- input (
TileTensor[input_type, Engine=input.Engine, address_space=input.address_space, linear_idx_type=input.linear_idx_type]): 4D NHWC input activation tensor of shape[batch, H, W, C_in]. - filter (
TileTensor[filter_type, Engine=filter.Engine, address_space=filter.address_space, linear_idx_type=filter.linear_idx_type]): 4D filter tensor;[R, S, C_in, C_out]or[C_out, C_in, R, S]depending onfilter_is_fcrs. - output (
TileTensor[output_type, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): 4D NHWC output tensor of shape[batch, H_out, W_out, C_out]to write into. - stride (
IndexList[Int(2)]): Spatial stride as[stride_h, stride_w]. - dilation (
IndexList[Int(2)]): Spatial dilation as[dilation_h, dilation_w]; must be1to dispatch. - symmetric_padding (
IndexList[Int(2)]): Symmetric padding as[pad_h, pad_w]. - num_groups (
Int): Group count; must be1to dispatch. - ctx (
DeviceContext): Device context used to enqueue kernels and allocate scratch buffers. - epilogue_fn (
EpilogueFnType): SIMD elementwise epilogue applied to each output element in 4D(batch, h, w, channel)coordinates; used only whenhas_epilogue_fnis true. - compute_fn (
ComputeFnType): SIMD elementwise transform of each output element in 4D(batch, h, w, channel)coordinates; used only whenhas_compute_fnis true.
Returns: