IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MatmulTileWriter

struct MatmulTileWriter[dtype: DType, tensor_layout: TensorLayout, output_engine: TensorEngine, linear_idx_type: DType, smem_tile_layout: TensorLayout, //, *, BM: Int, BN: Int, swizzle: TensorMapSwizzle, wgmma_shape: Tuple[Int, Int, Int], num_consumer: Int = Int(1), use_tma_store: Bool = False, swapAB: Bool = False]

Writes WGMMA accumulator register tiles to global memory for SM90 matmul.

Provides two write paths: TMA store (shared memory → global via hardware async copies) and thread-wise store (register → global directly). Handles swizzled shared memory layouts, optional elementwise epilogues, and A/B swap when the problem uses the swapAB small-M strategy.

Parameters​

  • ​dtype (DType): Element type of the output tensor and shared memory tile (inferred).
  • ​tensor_layout (TensorLayout): Memory layout of the output tensor in global memory (inferred).
  • ​output_engine (TensorEngine): Engine backing the output tensor in global memory (inferred).
  • ​linear_idx_type (DType): Integer type used for linear index arithmetic into the output tensor (inferred).
  • ​smem_tile_layout (TensorLayout): Memory layout of the shared memory tile used to stage output before storing to global memory; also defines the workgroup tile dimensions WG_BM and WG_BN (inferred).
  • ​BM (Int): Row (M) dimension of the matmul output tile per block.
  • ​BN (Int): Column (N) dimension of the matmul output tile per block.
  • ​swizzle (TensorMapSwizzle): TMA swizzle mode applied to shared memory for TMA async stores.
  • ​wgmma_shape (Tuple[Int, Int, Int]): Shape of each WGMMA instruction as (M, N, K).
  • ​num_consumer (Int): Number of consumer warp groups sharing the output tile (defaults to 1).
  • ​use_tma_store (Bool): Whether to use TMA async stores for shared memory to global memory copies (defaults to False).
  • ​swapAB (Bool): Whether to swap the A and B operand roles for the small-M strategy, transposing the tile and block coordinate mapping (defaults to False).

Fields​

  • ​tensor (MatmulTileWriter[BM=BM, BN=BN, swizzle=swizzle, wgmma_shape=wgmma_shape, num_consumer=num_consumer, use_tma_store=use_tma_store, swapAB=swapAB].CTensorType):
  • ​smem_tile (TileTensor[dtype, smem_tile_layout, MutAnyOrigin, address_space=AddressSpace.SHARED]):
  • ​warp_group_thread_idx (Int):
  • ​local_warp_group_idx (Int):
  • ​local_thread_idx (Int):
  • ​block_y (Int):
  • ​block_x (Int):

Implemented traits​

AnyType, Copyable, Deinitable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

comptime members​

CTensorType​

comptime CTensorType = TileTensor[dtype, tensor_layout, MutAnyOrigin, Engine=output_engine, linear_idx_type=linear_idx_type]

frag_size​

comptime frag_size = ((SIMD((load_from_mem wgmma_shape.__getitem_param__[Int(0)]())) * SIMD((load_from_mem wgmma_shape.__getitem_param__[Int(1)]()))) // _resolve_warpgroup_size())

lambda_type​

comptime lambda_type = def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](Coord[**?], mut SIMD[dtype, width]) -> None

N​

comptime N = tensor_layout.static_shape[Int(1)]

num_consumer_threads​

comptime num_consumer_threads = (num_consumer * _resolve_warpgroup_size())

num_m_mmas​

comptime num_m_mmas = ((BM // SIMD((load_from_mem wgmma_shape.__getitem_param__[Int(0)]()))) // num_consumer)

num_n_mmas​

comptime num_n_mmas = (BN // SIMD((load_from_mem wgmma_shape.__getitem_param__[Int(1)]())))

simd_size​

comptime simd_size = simd_width_of[dtype]()

WG_BM​

comptime WG_BM = smem_tile_layout.static_shape[Int(0)]

WG_BN​

comptime WG_BN = smem_tile_layout.static_shape[Int(1)]

Methods​

__init__​

def __init__(tensor: TileTensor[dtype, tensor_layout, MutAnyOrigin, Engine=output_engine, linear_idx_type=linear_idx_type], smem_tile: TileTensor[dtype, smem_tile_layout, MutAnyOrigin, address_space=AddressSpace.SHARED], warp_group_thread_idx: Int, local_warp_group_idx: Int, local_thread_idx: Int, block_y: Int, block_x: Int) -> Self

write_tile​

def write_tile[tma_tile_shape: Coord[*tma_tile_shape.element_types], tma_desc_shape: Coord[*tma_desc_shape.element_types], accum_type: DType, reg_tile_layout: TensorLayout, EpilogueFnType: ElementwiseEpilogueFn = inflated$def[comptime[dtype: DType, width: SIMDLength, *, alignment: Int] #kgen.func.symbol<@linalg::@utils::@"_no_epilogue_body[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>], ComputeFnType: ElementwiseComputeFn = inflated$def[comptime[dtype: DType, width: SIMDLength, *, alignment: Int] #kgen.func.symbol<@linalg::@utils::@"identity_compute_fn[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>], //, *, has_epilogue_fn: Bool = False, has_compute_fn: Bool = False](self, tma_op: TMATensorTile[dtype, tma_tile_shape, tma_desc_shape], reg_tile: TileTensor[accum_type, reg_tile_layout, Engine=reg_tile.Engine, address_space=AddressSpace.LOCAL, linear_idx_type=reg_tile.linear_idx_type], epilogue_fn: EpilogueFnType = no_epilogue_fn, compute_fn: ComputeFnType = no_compute_fn)

Write output from registers to global memory.

Selects optimized st.matrix path for bf16 when constraints are met, otherwise uses general register-to-global path.

Parameters:

  • ​tma_tile_shape (Coord[*element_types]): Shape of each TMA store tile per async copy.
  • ​tma_desc_shape (Coord[*element_types]): Full shape of the TMA tensor descriptor.
  • ​accum_type (DType): Data type of the WGMMA accumulator register tile.
  • ​reg_tile_layout (TensorLayout): Memory layout of the accumulator register tile.
  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn (inferred).
  • ​ComputeFnType (ElementwiseComputeFn): Type of compute_fn (inferred).
  • ​has_epilogue_fn (Bool): Whether epilogue_fn stores the output.
  • ​has_compute_fn (Bool): Whether compute_fn maps each output before it is stored.

Args:

Was this page helpful?