For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
TensorCoreAsync
struct TensorCoreAsync[c_type: DType, a_type: DType, b_type: DType, mma_shape: Coord[*mma_shape.element_types], /, a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_NONE, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_NONE, transpose_b: Bool = False]
High-performance asynchronous tensor core operations for matrix multiplication.
This struct provides methods for utilizing NVIDIA's Tensor Cores for asynchronous matrix multiplication operations, with support for various data types and swizzling configurations.
Parameters
- c_type (
DType): Data type of the output matrix C. - a_type (
DType): Data type of the input matrix A. - b_type (
DType): Data type of the input matrix B. - mma_shape (
Coord[*element_types]): Dimensions for the matrix multiply-accumulate (MMA) operation as [M, N, K]. - a_swizzle (
TensorMapSwizzle): Swizzling mode for matrix A (default: SWIZZLE_NONE). - b_swizzle (
TensorMapSwizzle): Swizzling mode for matrix B (default: SWIZZLE_NONE). - transpose_b (
Bool): Whether to transpose matrix B (default: False).
Implemented traits
AnyType,
Defaultable,
Deinitable,
Movable
Methods
__init__
def __init__(out self)
Initialize the TensorCoreAsync instance.
Ensures that the provided MMA shape is supported.
Note:
Fails to compile if mma_shape is not supported.
wgmma
static def wgmma[num_warp_groups: Int = Int(1), scale_c: Int = Int(1), scale_a: Int = Int(1), scale_b: Int = Int(1), num_k_iters: Optional[Int] = None](a_smem_tile: TileTensor[a_type, Engine=a_smem_tile.Engine, address_space=AddressSpace.SHARED, linear_idx_type=a_smem_tile.linear_idx_type], b_smem_tile: TileTensor[b_type, Engine=b_smem_tile.Engine, address_space=AddressSpace.SHARED, linear_idx_type=b_smem_tile.linear_idx_type], c_reg_tile: TileTensor[c_type, address_space=AddressSpace.LOCAL, linear_idx_type=c_reg_tile.linear_idx_type], wg_idx: Int = Int(0)) where (Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))) == Int(2)) if (c_reg_tile.LayoutType.rank == Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))))) else (c_reg_tile.LayoutType.rank == Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))))
Performs Hopper WGMMA with native shared-memory and register tiles.
Descriptor metadata keeps the exact nested shape and stride the descriptor formulas expect; the tensors stay native. C must be a contiguous row-major tile with one row per MMA and one scalar per fragment register, matching the tuple lane order.
Parameters:
- num_warp_groups (
Int): Number of warp groups to distribute work across (default: 1). - scale_c (
Int): Scale factor for matrix C. Valid values are 1 or 0 (default: 1). - scale_a (
Int): Scale factor for matrix A. Valid values are 1 or -1 (default: 1). - scale_b (
Int): Scale factor for matrix B. Valid values are 1 or -1 (default: 1). - num_k_iters (
Optional[Int]): Number of iterations for the K dimension. This is useful to save computation when we pad shared memory. (default: None, which is the A tile's K extent overmma_shape[2]).
Args:
- a_smem_tile (
TileTensor[a_type, Engine=a_smem_tile.Engine, address_space=AddressSpace.SHARED, linear_idx_type=a_smem_tile.linear_idx_type]): Matrix A in shared memory. - b_smem_tile (
TileTensor[b_type, Engine=b_smem_tile.Engine, address_space=AddressSpace.SHARED, linear_idx_type=b_smem_tile.linear_idx_type]): Matrix B in shared memory. - c_reg_tile (
TileTensor[c_type, address_space=AddressSpace.LOCAL, linear_idx_type=c_reg_tile.linear_idx_type]): Output matrix C in register memory. - wg_idx (
Int): Warp group index for multi-warp group scenarios (default: 0).
static def wgmma(a_frag_tile: TileTensor[a_type, Engine=a_frag_tile.Engine, address_space=AddressSpace.LOCAL, linear_idx_type=a_frag_tile.linear_idx_type], b_smem_tile: TileTensor[b_type, Engine=b_smem_tile.Engine, address_space=AddressSpace.SHARED, linear_idx_type=b_smem_tile.linear_idx_type], c_reg_tile: TileTensor[c_type, address_space=AddressSpace.LOCAL, linear_idx_type=c_reg_tile.linear_idx_type]) where (Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))) == Int(2)) if (a_frag_tile.LayoutType.rank == Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))))) else (a_frag_tile.LayoutType.rank == Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(a_frag_tile.LayoutType.__shape_types), [idx: __mlir_type.index] a_frag_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))))) and (Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))) == Int(2)) if (c_reg_tile.LayoutType.rank == Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))))) else (c_reg_tile.LayoutType.rank == Int(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(#kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))), [idx: __mlir_type.index] #kgen.param_list.concat(#kgen.param_list.tabulate(len(c_reg_tile.LayoutType.__shape_types), [idx: __mlir_type.index] c_reg_tile.LayoutType.__shape_types[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType))[idx]._ParamListType)))))
Performs Hopper WGMMA with A in registers and B in shared memory.
Row m_mma + k_mma * num_m_mmas of a_frag_tile holds the A
fragment for that MMA, and row n_mma * num_m_mmas + m_mma of
c_reg_tile holds the accumulator, both as contiguous row-major
scalar register storage.
Args:
- a_frag_tile (
TileTensor[a_type, Engine=a_frag_tile.Engine, address_space=AddressSpace.LOCAL, linear_idx_type=a_frag_tile.linear_idx_type]): Matrix A in register memory, one row per A fragment. - b_smem_tile (
TileTensor[b_type, Engine=b_smem_tile.Engine, address_space=AddressSpace.SHARED, linear_idx_type=b_smem_tile.linear_idx_type]): Matrix B in shared memory. - c_reg_tile (
TileTensor[c_type, address_space=AddressSpace.LOCAL, linear_idx_type=c_reg_tile.linear_idx_type]): Output matrix C in register memory, one row per MMA.
arrive
static def arrive()
Ensures memory consistency by creating a fence for WGMMA operations.
This method should be called before committing a group to ensure all shared memory accesses are properly aligned and visible.
commit_group
static def commit_group()
Commits the current warp group for execution.
This synchronizes the warp group and commits all pending WGMMA operations that have been previously issued.
wait_group
static def wait_group[group: Int = Int(0)]()
Waits for the completion of a specific warp group's operations.
This method blocks until all WGMMA operations from the specified group are complete.
Parameters:
- group (
Int): The group ID to wait for (default: 0).