IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

gemm_mma_cpasync

def gemm_mma_cpasync[pdl_level: PDLLevel = PDLLevel(), tile_k: Int = Int(128), swapAB: Bool = False](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], act: TileTensor[Engine=act.Engine, address_space=act.address_space, linear_idx_type=act.linear_idx_type], weight: TileTensor[Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], gemm_m: Int, gemm_k: Int, gemm_n: Int, batch_size: Int, ctx: DeviceContext)

Launch the batched GEMM tensor-core kernel.

C[gemm_m, gemm_n] = act[gemm_m, K] x weight[gemm_n, K]^T.

The caller always passes act/weight/c with the same shapes regardless of swapAB, and c is always written as a row-major [gemm_m, gemm_n] ([M, N]) buffer. swapAB only changes the internal tiling: when True the weight is fed to the A (free-streaming) operand slot and the activation to the B slot (so the grid tiles N by tile_m and M by tile_n), and the epilogue transposes the store back into the row-major [M, N] buffer. This makes the large weight the producer-independent operand for PDL overlap at small M (decode), while keeping the output layout identical.

Parameters:

  • ​pdl_level (PDLLevel): Programmatic dependent launch level for PDL barriers.
  • ​tile_k (Int): K-dimension tile size for the MMA kernel (defaults to 128).
  • ​swapAB (Bool): When True, feeds the weight to the A operand slot and the activation to the B slot for PDL overlap at small M.

Args:

def gemm_mma_cpasync[EpilogueFnType: ElementwiseEpilogueFn, //, pdl_level: PDLLevel = PDLLevel(), tile_k: Int = Int(128), swapAB: Bool = False](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], act: TileTensor[Engine=act.Engine, address_space=act.address_space, linear_idx_type=act.linear_idx_type], weight: TileTensor[Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], gemm_m: Int, gemm_k: Int, gemm_n: Int, batch_size: Int, epilogue_fn: EpilogueFnType, ctx: DeviceContext)

Launches the batched GEMM tensor-core kernel, writing act @ weight^T through epilogue_fn.

Parameters:

  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn (inferred).
  • ​pdl_level (PDLLevel): Programmatic dependent launch level for PDL barriers.
  • ​tile_k (Int): K-dimension tile size for the MMA kernel (defaults to 128).
  • ​swapAB (Bool): When True, feeds the weight to the A operand slot and the activation to the B slot for PDL overlap at small M.

Args:

def gemm_mma_cpasync[ComputeFnType: ElementwiseComputeFn, //, pdl_level: PDLLevel = PDLLevel(), tile_k: Int = Int(128), swapAB: Bool = False](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], act: TileTensor[Engine=act.Engine, address_space=act.address_space, linear_idx_type=act.linear_idx_type], weight: TileTensor[Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], gemm_m: Int, gemm_k: Int, gemm_n: Int, batch_size: Int, compute_fn: ComputeFnType, ctx: DeviceContext)

Launches the batched GEMM tensor-core kernel, storing compute_fn(idx, act @ weight^T) into c.

Parameters:

  • ​ComputeFnType (ElementwiseComputeFn): Type of compute_fn (inferred).
  • ​pdl_level (PDLLevel): Programmatic dependent launch level for PDL barriers.
  • ​tile_k (Int): K-dimension tile size for the MMA kernel (defaults to 128).
  • ​swapAB (Bool): When True, feeds the weight to the A operand slot and the activation to the B slot for PDL overlap at small M.

Args:

Was this page helpful?