IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

BlackwellMatmulSM100FallbackKernel

struct BlackwellMatmulSM100FallbackKernel[a_type: DType, b_type: DType, c_type: DType, c_layout: TensorLayout, c_engine: TensorEngine, block_tile_shape: Tuple[Int, Int, Int], mma_shape: Tuple[Int, Int, Int], transpose_b: Bool = True, cluster_shape: Coord[Int, Int, Int] = Coord(Int(1), Int(1), Int(1)), a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, num_threads: Int = Int(128)]

Simple fallback matmul kernel for SM100 (B200).

This kernel is used when the warp-specialized kernel is not applicable, such as for small problem sizes or unsupported configurations.

Unlike the main BlackwellMatmulSM100Kernel, this uses:

  • Single warp approach (no warp specialization)
  • Basic barrier synchronization (no CLC scheduling)
  • Direct TileTensor output (no TMA for C)
  • Simpler pipeline with single buffer

Parameters​

  • ​a_type (DType): Element type of the A input matrix.
  • ​b_type (DType): Element type of the B input matrix.
  • ​c_type (DType): Element type of the C output matrix.
  • ​c_layout (TensorLayout): Memory layout of the C output tensor in global memory, used for output tiling and static stride computation.
  • ​c_engine (TensorEngine): Engine of the C output tensor in global memory.
  • ​block_tile_shape (Tuple[Int, Int, Int]): Block tile dimensions (BM, BN, BK) for CTA-level tiling of the output and reduction dimensions.
  • ​mma_shape (Tuple[Int, Int, Int]): MMA instruction dimensions (MMA_M, MMA_N, MMA_K) for the tensor core operation.
  • ​transpose_b (Bool): Whether B is stored transposed (K-major) (defaults to True).
  • ​cluster_shape (Coord[Int, Int, Int]): Thread block cluster dimensions used for LLVM cluster metadata (defaults to (1, 1, 1)).
  • ​a_swizzle (TensorMapSwizzle): Swizzle pattern for A shared memory tiles (defaults to TensorMapSwizzle.SWIZZLE_128B).
  • ​b_swizzle (TensorMapSwizzle): Swizzle pattern for B shared memory tiles (defaults to TensorMapSwizzle.SWIZZLE_128B).
  • ​num_threads (Int): Number of threads per CTA; must be 128 or 256 (defaults to 128).

Implemented traits​

AnyType, Deinitable, Movable

comptime members​

a_size​

comptime a_size = (BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BM * BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK)

a_smem_layout_typed​

comptime a_smem_layout_typed = Layout(Coord(Coord(ComptimeInt(), ComptimeInt()), Coord(ComptimeInt(), ComptimeInt())), Coord(Coord(ComptimeInt(), ComptimeInt()), Coord(ComptimeInt(), ComptimeInt())))

a_swizzle_elems​

comptime a_swizzle_elems = (a_swizzle.bytes() // size_of[a_type]())

accum_type​

comptime accum_type = get_accum_type[a_type]()

ADescLayout​

comptime ADescLayout = static_row_major[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BM, (a_swizzle.bytes() // size_of[a_type]())]

ATile​

comptime ATile = SMemTile[a_type, BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].a_smem_layout_typed]

ATileLayout​

comptime ATileLayout = static_row_major[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BM, BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK]

ATmaOp​

comptime ATmaOp = TmaOpType[a_type, Layout[TypeList[ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BM], ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK]](), TypeList[ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK], ComptimeInt[Int(1)]]()], Layout[TypeList[ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BM], ComptimeInt[(a_swizzle.bytes() // size_of[a_type]())]](), TypeList[ComptimeInt[(a_swizzle.bytes() // size_of[a_type]())], ComptimeInt[Int(1)]]()]]

b_size​

comptime b_size = (BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BN * BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK)

b_smem_layout_typed​

comptime b_smem_layout_typed = Layout(Coord(Coord(ComptimeInt(), ComptimeInt()), Coord(ComptimeInt(), ComptimeInt())), Coord(Coord(ComptimeInt(), ComptimeInt()), Coord(ComptimeInt(), ComptimeInt())))

b_swizzle_elems​

comptime b_swizzle_elems = (b_swizzle.bytes() // size_of[b_type]())

BDescLayout​

comptime BDescLayout = static_row_major[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BN, (b_swizzle.bytes() // size_of[b_type]())]

BK​

comptime BK = (load_from_mem block_tile_shape.__getitem_param__[Int(2)]())

BM​

comptime BM = (load_from_mem block_tile_shape.__getitem_param__[Int(0)]())

BN​

comptime BN = (load_from_mem block_tile_shape.__getitem_param__[Int(1)]())

BTile​

comptime BTile = SMemTile[b_type, BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].b_smem_layout_typed]

BTileLayout​

comptime BTileLayout = static_row_major[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BN, BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK]

BTmaOp​

comptime BTmaOp = TmaOpType[b_type, Layout[TypeList[ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BN], ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK]](), TypeList[ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK], ComptimeInt[Int(1)]]()], Layout[TypeList[ComptimeInt[BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BN], ComptimeInt[(b_swizzle.bytes() // size_of[b_type]())]](), TypeList[ComptimeInt[(b_swizzle.bytes() // size_of[b_type]())], ComptimeInt[Int(1)]]()]]

c_frag_size​

comptime c_frag_size = ((BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].MMA_M * BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].MMA_N) // num_threads)

CGmemStrideLayout​

comptime CGmemStrideLayout = Layout[TypeList[ComptimeInt[c_layout.static_stride[Int(0)]], ComptimeInt[Int(1)]](), TypeList[ComptimeInt[Int(1)], ComptimeInt[Int(1)]]()]

max_tmem_cols​

comptime max_tmem_cols = 512

MMA_K​

comptime MMA_K = (load_from_mem mma_shape.__getitem_param__[Int(2)]())

MMA_M​

comptime MMA_M = (load_from_mem mma_shape.__getitem_param__[Int(0)]())

MMA_N​

comptime MMA_N = (load_from_mem mma_shape.__getitem_param__[Int(1)]())

num_k_mmas​

comptime num_k_mmas = (BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BK // BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].MMA_K)

num_m_mmas​

comptime num_m_mmas = (BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BM // BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].MMA_M)

num_n_mmas​

comptime num_n_mmas = (BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].BN // BlackwellMatmulSM100FallbackKernel[a_type, b_type, c_type, c_layout, c_engine, block_tile_shape, mma_shape, transpose_b, cluster_shape, a_swizzle, b_swizzle, num_threads].MMA_N)

static_N​

comptime static_N = c_layout.static_stride[Int(0)]

Methods​

validate_constraints​

static def validate_constraints()

Validate compile-time constraints for this kernel configuration.

run​

static def run(a_tma_op: TMATensorTile[a_type, _to_coord[Layout[TypeList[ComptimeInt[Self.BM], ComptimeInt[Self.BK]](), TypeList[ComptimeInt[Self.BK], ComptimeInt[Int(1)]]()]](), _to_coord[Int(2), Layout[TypeList[ComptimeInt[Self.BM], ComptimeInt[(a_swizzle.bytes() // size_of[a_type]())]](), TypeList[ComptimeInt[(a_swizzle.bytes() // size_of[a_type]())], ComptimeInt[Int(1)]]()]]()], b_tma_op: TMATensorTile[b_type, _to_coord[Layout[TypeList[ComptimeInt[Self.BN], ComptimeInt[Self.BK]](), TypeList[ComptimeInt[Self.BK], ComptimeInt[Int(1)]]()]](), _to_coord[Int(2), Layout[TypeList[ComptimeInt[Self.BN], ComptimeInt[(b_swizzle.bytes() // size_of[b_type]())]](), TypeList[ComptimeInt[(b_swizzle.bytes() // size_of[b_type]())], ComptimeInt[Int(1)]]()]]()], c: TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine], num_iters: Int32)

Run the fallback matmul kernel.

Args:

run_with_compute_fn​

static def run_with_compute_fn[ComputeFnType: ElementwiseComputeFn](a_tma_op: TMATensorTile[a_type, _to_coord[Layout[TypeList[ComptimeInt[Self.BM], ComptimeInt[Self.BK]](), TypeList[ComptimeInt[Self.BK], ComptimeInt[Int(1)]]()]](), _to_coord[Int(2), Layout[TypeList[ComptimeInt[Self.BM], ComptimeInt[(a_swizzle.bytes() // size_of[a_type]())]](), TypeList[ComptimeInt[(a_swizzle.bytes() // size_of[a_type]())], ComptimeInt[Int(1)]]()]]()], b_tma_op: TMATensorTile[b_type, _to_coord[Layout[TypeList[ComptimeInt[Self.BN], ComptimeInt[Self.BK]](), TypeList[ComptimeInt[Self.BK], ComptimeInt[Int(1)]]()]](), _to_coord[Int(2), Layout[TypeList[ComptimeInt[Self.BN], ComptimeInt[(b_swizzle.bytes() // size_of[b_type]())]](), TypeList[ComptimeInt[(b_swizzle.bytes() // size_of[b_type]())], ComptimeInt[Int(1)]]()]]()], c: TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine], num_iters: Int32, compute_fn: ComputeFnType)

Run the fallback matmul kernel with a compute epilogue value.

Parameters:

Args:

Was this page helpful?