For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

naive_grouped_matmul_kernel

def naive_grouped_matmul_kernel[c_type: DType, a_type: DType, b_type: DType, CLayout: TensorLayout, ALayout: TensorLayout, BLayout: TensorLayout, AOffsetsLayout: TensorLayout, ExpertIdsLayout: TensorLayout, *, elementwise_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None] = None, a_plane_splits: IndexList[Int(2)] = Index[Int, Int](Int(0), Int(0))](c: TileTensor[c_type, CLayout, MutAnyOrigin], a: TileTensor[a_type, ALayout, ImmUnsafeAnyOrigin], b: TileTensor[b_type, BLayout, ImmUnsafeAnyOrigin], a_offsets: TileTensor[DType.uint32, AOffsetsLayout, ImmUnsafeAnyOrigin], expert_ids: TileTensor[DType.int32, ExpertIdsLayout, ImmUnsafeAnyOrigin])

Computes one element per thread of the grouped matmul product C[a_offsets[z]:a_offsets[z+1], :] = A[...] @ B[expert_ids[z], :, :].T for each active expert z, with an optional elementwise epilogue.

Skips the matmul for expert == -1 (inactive LoRA blocks) but still invokes the elementwise lambda when provided.