For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
test_matmul_sm90
def test_matmul_sm90[MType: CoordLike, NType: CoordLike, KType: CoordLike, ComputeFnType: ElementwiseComputeFn = inflated$def[comptime[dtype: DType, width: SIMDLength, *, alignment: Int] #kgen.func.symbol<@linalg::@utils::@"identity_compute_fn[::DType,::SIMDLength,::SIMD[DType.int, 1],KGENParamList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike],::TypeList[::Intable & ::Defaultable & ::RegisterPassable & ::TrivialRegisterPassable & ::Writable & ::AnyType & ::Copyable & ::ImplicitlyCopyable & ::Deinitable & ::Movable & ::CoordLike, $3]](::Coord[$3, $4],::SIMD[$0, $1])"<:!lit.struct<@std::@builtin::@dtype::@DType> dtype, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> width, :!lit.struct<@std::@simd::@SIMD<:!lit.struct<@std::@builtin::@dtype::@DType> {:dtype index}, :!lit.struct<@std::@builtin::@simd_length::@SIMDLength> {1}>> alignment, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?, :!lit.struct<@std::@builtin::@variadics::@TypeList<:meta<!lit.trait<@std::@traits::@anytype::@AnyType>> trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>, :param_list<trait<@std::@builtin::@int::@Intable, @std::@builtin::@value::@Defaultable, @std::@builtin::@value::@RegisterPassable, @std::@builtin::@value::@TrivialRegisterPassable, @std::@format::@__init__::@Writable, @std::@traits::@anytype::@AnyType, @std::@traits::@copyable::@Copyable, @std::@traits::@copyable::@ImplicitlyCopyable, @std::@traits::@deinitable::@Deinitable, @std::@traits::@movable::@Movable, @std::@utils::@coord::@CoordLike>> *?>> *?>>], //, a_type: DType, b_type: DType, c_type: DType, cluster_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], wgmma_shape: Tuple[Int, Int, Int], num_consumer: Int = Int(1), num_pipeline_stages: Int = Int(4), transpose_b: Bool = True, partitioned_multicast: Bool = False, grid_shape: Optional[Tuple[Int, Int]] = None, use_tma_store: Bool = False, schedule: MatmulSchedule = MatmulSchedule.NONE, default_epilogue: Bool = False, has_compute_fn: Bool = False, measure_threshold: Optional[Float64] = None, backend: Backend = Backend.CUBLAS, k_group_size: Int = Int(1)](ctx: DeviceContext, m: MType, n: NType, k: KType, compute_fn: ComputeFnType = no_compute_fn)
Runs an SM90 warp-specialized matmul and validates the result against a vendor reference.
Allocates host and device buffers for the operands, initializes them with random data, launches the warp-specialized GEMM with multicasting, and compares the output against a vendor (cuBLAS) reference using an elementwise tolerance check.
Parameters:
- MType (
CoordLike): Coordinate-like type encoding the M dimension extent (inferred). - NType (
CoordLike): Coordinate-like type encoding the N dimension extent (inferred). - KType (
CoordLike): Coordinate-like type encoding the K dimension extent (inferred). - ComputeFnType (
ElementwiseComputeFn): Type ofcompute_fn(inferred). - a_type (
DType): Element dtype of the left-hand operand. - b_type (
DType): Element dtype of the right-hand operand. - c_type (
DType): Element dtype of the output matrix. - cluster_shape (
Tuple[Int, Int, Int]): Thread block cluster shape as(N, M, K). - block_tile_shape (
Tuple[Int, Int, Int]): Block tile shape as(BM, BN, BK). - wgmma_shape (
Tuple[Int, Int, Int]): WGMMA instruction shape as(M, N, K). - num_consumer (
Int): Number of consumer warps in the warp-specialized pipeline. - num_pipeline_stages (
Int): Number of software pipeline stages. - transpose_b (
Bool): Whether the right-hand operand is stored transposed. - partitioned_multicast (
Bool): Whether to use partitioned instead of broadcast multicast. - grid_shape (
Optional[Tuple[Int, Int]]): Optional explicit grid shape override. - use_tma_store (
Bool): Whether to use TMA store for the epilogue. - schedule (
MatmulSchedule): Tile scheduler hint controlling launch order. - default_epilogue (
Bool): Whether to store the output through an epilogue closure instead of the kernel's own store. - has_compute_fn (
Bool): Whether the kernel storescompute_fn(idx, a @ b); takes precedence overdefault_epilogue. - measure_threshold (
Optional[Float64]): Optional relative-difference threshold for a measured assertion. - backend (
Backend): Vendor backend used for the reference matmul. - k_group_size (
Int): Group size used for grouped matmul along the K dimension.
Args:
- ctx (
DeviceContext): Device context used for device allocations and kernel dispatch. - m (
MType): Number of rows of the output matrix (M dimension). - n (
NType): Number of columns of the output matrix (N dimension). - k (
KType): Contraction dimension shared by the operands (K dimension). - compute_fn (
ComputeFnType): Maps each output index and value to the value to store whenhas_compute_fnis set.