For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
enqueue_apple_matmul_split_k
def enqueue_apple_matmul_split_k[EpilogueFnType: ElementwiseEpilogueFn = inflated$def[_no_epilogue_body[_, _, _, _, _]], ComputeFnType: ElementwiseComputeFn = inflated$def[identity_compute_fn[_, _, _, _, _]], //, in_type: DType, c_type: DType = .float32, transpose_b: Bool = False, has_epilogue_fn: Bool = False, has_compute_fn: Bool = False](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[in_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[in_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], ctx: DeviceContextBase, num_splits_hint: Int = Int(4), *, epilogue_fn: EpilogueFnType = no_epilogue_fn, compute_fn: ComputeFnType = no_compute_fn)
Split-K Apple M5 matmul: partition K, accumulate partials, reduce.
num_splits_hint is an upper bound; the actual split count is capped so no
split is empty (actual_splits = ceil(num_strips / strips_per_split) where
strips_per_split = ceil(num_strips / num_splits_hint)). Best for large-K,
small-M*N shapes where the single-pass kernel under-occupies the GPU.
Parameters:
- EpilogueFnType (
ElementwiseEpilogueFn): Type ofepilogue_fn(inferred). - ComputeFnType (
ElementwiseComputeFn): Type ofcompute_fn(inferred). - in_type (
DType): A/B element type (fp16, bf16, fp32). - c_type (
DType): Output element type (fp16, bf16, fp32). Accumulation is fp32 (defaults tofloat32). - transpose_b (
Bool): If True, B is(N, K)row-major (viewed ascol_major(K, N)); otherwise B is(K, N)row-major (defaults to False). - has_epilogue_fn (
Bool): Whether the reduce stores each output throughepilogue_fnin place ofc(defaults to False). - has_compute_fn (
Bool): Whether the reduce maps each output throughcompute_fnbefore it is stored toc(defaults to False).
Args:
- c (
TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output matrix(M, N)row-major;MandNderive from its dims. - a (
TileTensor[in_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): A operand matrix(M, K)row-major;Kderives froma.dim[1]. - b (
TileTensor[in_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): B operand matrix,(K, N)fortranspose_b=Falseor(N, K)fortranspose_b=True. - ctx (
DeviceContextBase): Device context to enqueue the kernels on; must be Apple M5 (compute_capability == 5). - num_splits_hint (
Int): Upper bound on the K-axis split count; the actual split count is capped so no split is empty (defaults to 4). - epilogue_fn (
EpilogueFnType): Epilogue that stores each output. - compute_fn (
ComputeFnType): Compute epilogue applied to each output.
Raises:
If the attached GPU is not Apple M5 (compute_capability != 5).