For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
multistage_gemm
def multistage_gemm[c_type: DType, a_type: DType, b_type: DType, //, *, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b]](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], ctx: DeviceContextBase)
TileTensor overload of multistage_gemm. Dispatches to a GEMM kernel over the native TileTensor operands; the device-side pipeline bridges to legacy LayoutTensorIter views internally (the multistage pipeline is iterator-driven and has no TileTensor equivalent yet).
Parameters:
- c_type (
DType): DType of the output tilecelements (inferred). - a_type (
DType): DType of the input tileaelements (inferred). - b_type (
DType): DType of the input tilebelements (inferred). - transpose_b (
Bool): Whetherbis accessed transposed, so its rowyis read asb[y, i]instead ofb[i, y]. - config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): Compile-timeMatmulConfigselecting the block tile, warp tile, MMA shape, and pipeline stages for the kernel.
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output tile of shape(M, N)receiving the matmul result. - a (
TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input tile of shape(M, K). - b (
TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Input tile of shape(K, N), or(N, K)whentranspose_bis set. - ctx (
DeviceContextBase): Device context used to enqueue the kernel.
def multistage_gemm[c_type: DType, a_type: DType, b_type: DType, EpilogueFnType: ElementwiseEpilogueFn, //, *, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b]](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], epilogue_fn: EpilogueFnType, ctx: DeviceContextBase)
Runs multistage_gemm, storing the output through an epilogue closure value.
Parameters:
- c_type (
DType): DType of the output tilecelements (inferred). - a_type (
DType): DType of the input tileaelements (inferred). - b_type (
DType): DType of the input tilebelements (inferred). - EpilogueFnType (
ElementwiseEpilogueFn): Type ofepilogue_fn(inferred). - transpose_b (
Bool): Whetherbis accessed transposed, so its rowyis read asb[y, i]instead ofb[i, y]. - config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): Compile-timeMatmulConfigselecting the block tile, warp tile, MMA shape, and pipeline stages for the kernel.
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output tile of shape(M, N). Only its shape is read. - a (
TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input tile of shape(M, K). - b (
TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Input tile of shape(K, N), or(N, K)whentranspose_bis set. - epilogue_fn (
EpilogueFnType): Stores each output element. - ctx (
DeviceContextBase): Device context used to enqueue the kernel.
def multistage_gemm[c_type: DType, a_type: DType, b_type: DType, ComputeFnType: ElementwiseComputeFn, //, *, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b]](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], compute_fn: ComputeFnType, ctx: DeviceContextBase)
Runs multistage_gemm, storing compute_fn(idx, a @ b) into c.
Parameters:
- c_type (
DType): DType of the output tilecelements (inferred). - a_type (
DType): DType of the input tileaelements (inferred). - b_type (
DType): DType of the input tilebelements (inferred). - ComputeFnType (
ElementwiseComputeFn): Type ofcompute_fn(inferred). - transpose_b (
Bool): Whetherbis accessed transposed, so its rowyis read asb[y, i]instead ofb[i, y]. - config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): Compile-timeMatmulConfigselecting the block tile, warp tile, MMA shape, and pipeline stages for the kernel.
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output tile of shape(M, N)receiving the result. - a (
TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input tile of shape(M, K). - b (
TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Input tile of shape(K, N), or(N, K)whentranspose_bis set. - compute_fn (
ComputeFnType): Maps each output index and value to the value to store. - ctx (
DeviceContextBase): Device context used to enqueue the kernel.
def multistage_gemm[c_type: DType, a_type: DType, b_type: DType, //, *, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b]](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], runtime_config: MatmulConfig[a_type, b_type, c_type, transpose_b], ctx: DeviceContextBase)
TileTensor overload of multistage_gemm with runtime config. Constrains c to mut=True because split_k_reduce requires a mutable output tensor.
Parameters:
- c_type (
DType): DType of the output tilecelements (inferred). - a_type (
DType): DType of the input tileaelements (inferred). - b_type (
DType): DType of the input tilebelements (inferred). - transpose_b (
Bool): Whetherbis accessed transposed, so its rowyis read asb[y, i]instead ofb[i, y]. - config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): Compile-timeMatmulConfigselecting the block tile, warp tile, MMA shape, and pipeline stages for the kernel.
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output tile of shape(M, N)receiving the matmul result; must be mutable becausesplit_k_reducewrites the reduced sum back into it. - a (
TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input tile of shape(M, K). - b (
TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Input tile of shape(K, N), or(N, K)whentranspose_bis set. - runtime_config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): RuntimeMatmulConfigcarrying the number of K partitions used for the split-K reduction path; whennum_k_partitionsis greater than 1 the kernel writes partial sums to a workspace and reduces them intoc. - ctx (
DeviceContextBase): Device context used to enqueue the kernel and allocate the split-K workspace.
def multistage_gemm[c_type: DType, a_type: DType, b_type: DType, EpilogueFnType: ElementwiseEpilogueFn, //, *, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b]](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], runtime_config: MatmulConfig[a_type, b_type, c_type, transpose_b], epilogue_fn: EpilogueFnType, ctx: DeviceContextBase)
Runs multistage_gemm with runtime config, storing the output through an epilogue closure value.
Parameters:
- c_type (
DType): DType of the output tilecelements (inferred). - a_type (
DType): DType of the input tileaelements (inferred). - b_type (
DType): DType of the input tilebelements (inferred). - EpilogueFnType (
ElementwiseEpilogueFn): Type ofepilogue_fn(inferred). - transpose_b (
Bool): Whetherbis accessed transposed, so its rowyis read asb[y, i]instead ofb[i, y]. - config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): Compile-timeMatmulConfigselecting the block tile, warp tile, MMA shape, and pipeline stages for the kernel.
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output tile of shape(M, N). Only its shape is read. - a (
TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input tile of shape(M, K). - b (
TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Input tile of shape(K, N), or(N, K)whentranspose_bis set. - runtime_config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): RuntimeMatmulConfigcarrying the number of K partitions used for the split-K reduction path. - epilogue_fn (
EpilogueFnType): Stores each output element. - ctx (
DeviceContextBase): Device context used to enqueue the kernel and allocate the split-K workspace.
def multistage_gemm[c_type: DType, a_type: DType, b_type: DType, ComputeFnType: ElementwiseComputeFn, //, *, transpose_b: Bool, config: MatmulConfig[a_type, b_type, c_type, transpose_b]](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], runtime_config: MatmulConfig[a_type, b_type, c_type, transpose_b], compute_fn: ComputeFnType, ctx: DeviceContextBase)
Runs multistage_gemm with runtime config, storing compute_fn(idx, a @ b) into c.
Parameters:
- c_type (
DType): DType of the output tilecelements (inferred). - a_type (
DType): DType of the input tileaelements (inferred). - b_type (
DType): DType of the input tilebelements (inferred). - ComputeFnType (
ElementwiseComputeFn): Type ofcompute_fn(inferred). - transpose_b (
Bool): Whetherbis accessed transposed, so its rowyis read asb[y, i]instead ofb[i, y]. - config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): Compile-timeMatmulConfigselecting the block tile, warp tile, MMA shape, and pipeline stages for the kernel.
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output tile of shape(M, N)receiving the result. - a (
TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input tile of shape(M, K). - b (
TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Input tile of shape(K, N), or(N, K)whentranspose_bis set. - runtime_config (
MatmulConfig[a_type, b_type, c_type, transpose_b]): RuntimeMatmulConfigcarrying the number of K partitions used for the split-K reduction path. - compute_fn (
ComputeFnType): Maps each output index and value to the value to store. - ctx (
DeviceContextBase): Device context used to enqueue the kernel and allocate the split-K workspace.