IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

batched_matmul_kernel_gpu

def batched_matmul_kernel_gpu[rank: Int, c_type: DType, a_type: DType, b_type: DType, CTensorType: TensorLayout, ATensorType: TensorLayout, BTensorType: TensorLayout, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b], EpilogueFnType: ElementwiseEpilogueFn = inflated$def[_no_epilogue_body[_, _, _, _, _]], has_epilogue_fn: Bool = False](c_tensor: TileTensor[c_type, CTensorType, MutAnyOrigin], a_tensor: TileTensor[a_type, ATensorType, ImmutAnyOrigin], b_tensor: TileTensor[b_type, BTensorType, ImmutAnyOrigin], c_buff_nd_shape: IndexList[rank], epilogue_fn: EpilogueFnType)

Computes a single batch slice of a batched matrix multiplication on the GPU by dispatching to the multistage GEMM kernel on NVIDIA or the AMDMatmul kernel on AMD hardware.

Parameters:

  • ​rank (Int): Rank of the original (un-collapsed) output tensor.
  • ​c_type (DType): Output tensor element dtype.
  • ​a_type (DType): LHS input tensor element dtype.
  • ​b_type (DType): RHS input tensor element dtype.
  • ​CTensorType (TensorLayout): Layout type of the output tensor.
  • ​ATensorType (TensorLayout): Layout type of the LHS input tensor.
  • ​BTensorType (TensorLayout): Layout type of the RHS input tensor.
  • ​transpose_b (Bool): Whether the RHS input is transposed.
  • ​config (MatmulConfig[a_type, b_type, c_type, transpose_b]): Matmul kernel configuration for the target hardware.
  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn.
  • ​has_epilogue_fn (Bool): Whether epilogue_fn stores the output in place of c_tensor (defaults to False).

Args:

Was this page helpful?