For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

gemv_split_k

def gemv_split_k[c_type: DType, a_type: DType, b_type: DType, c_layout: TensorLayout, a_layout: TensorLayout, b_layout: TensorLayout, c_engine: TensorEngine, a_engine: TensorEngine, b_engine: TensorEngine, simd_width: Int, tile_m: Int, tile_n: Int, num_threads: Int, unroll_factor: Int = Int(2), weight_non_temporal: Bool = True, elementwise_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None] = None, accum_type: DType = get_accum_type[c_type](), check_bounds_m: Bool = True, check_bounds_n: Bool = True, pdl_level: PDLLevel = PDLLevel()](output: TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine], act: TileTensor[a_type, a_layout, ImmutAnyOrigin, Engine=a_engine], weight: TileTensor[b_type, b_layout, ImmutAnyOrigin, Engine=b_engine], m: Int32, n: Int32, k: Int32)

GEMV with tiling in K dimension; see _gemv_split_k_impl.

Parameters:

• ​c_type (DType): Output element type.
• ​a_type (DType): Activation matrix element type.
• ​b_type (DType): Weight matrix element type.
• ​c_layout (TensorLayout): Layout descriptor for the output tensor.
• ​a_layout (TensorLayout): Layout descriptor for the activation matrix.
• ​b_layout (TensorLayout): Layout descriptor for the weight matrix.
• ​c_engine (TensorEngine): Engine of the output tensor.
• ​a_engine (TensorEngine): Engine of the activation matrix.
• ​b_engine (TensorEngine): Engine of the weight matrix.
• ​simd_width (Int): Number of elements per vectorized load.
• ​tile_m (Int): Number of output rows each thread accumulates.
• ​tile_n (Int): Number of weight rows each thread accumulates.
• ​num_threads (Int): Threads per block.
• ​unroll_factor (Int): K-loop unroll factor.
• ​weight_non_temporal (Bool): Whether to stream the weight matrix.
• ​elementwise_lambda_fn (Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None]): Optional epilogue applied to each output element.
• ​accum_type (DType): Accumulation precision type.
• ​check_bounds_m (Bool): Whether to guard M-tail rows.
• ​check_bounds_n (Bool): Whether to guard N-tail columns.
• ​pdl_level (PDLLevel): Programmatic dependent launch level.

Args:

• ​output (TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine]): Output tensor, shape (m, n), row-major.
• ​act (TileTensor[a_type, a_layout, ImmutAnyOrigin, Engine=a_engine]): Activation matrix, shape (m, k).
• ​weight (TileTensor[b_type, b_layout, ImmutAnyOrigin, Engine=b_engine]): Weight matrix, shape (n, k), row-major transposed B.
• ​m (Int32): Number of activation rows, output rows.
• ​n (Int32): Number of weight rows, output columns.
• ​k (Int32): Reduction dimension shared by activation and weight.