IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

gemv_split_k_compute_fn

def gemv_split_k_compute_fn[c_type: DType, a_type: DType, b_type: DType, c_layout: TensorLayout, a_layout: TensorLayout, b_layout: TensorLayout, c_engine: TensorEngine, a_engine: TensorEngine, b_engine: TensorEngine, ComputeFnType: ElementwiseComputeFn, simd_width: Int, tile_m: Int, tile_n: Int, num_threads: Int, unroll_factor: Int = Int(2), weight_non_temporal: Bool = True, accum_type: DType = get_accum_type[c_type](), check_bounds_m: Bool = True, check_bounds_n: Bool = True, pdl_level: PDLLevel = PDLLevel(), small_m_static_k: Bool = False](output: TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine], act: TileTensor[a_type, a_layout, ImmutAnyOrigin, Engine=a_engine], weight: TileTensor[b_type, b_layout, ImmutAnyOrigin, Engine=b_engine], m: Int32, n: Int32, k: Int32, compute_fn: ComputeFnType)

gemv_split_k that stores compute_fn applied to each output element.

Parameters:

  • ​c_type (DType): Output element type.
  • ​a_type (DType): Activation matrix element type.
  • ​b_type (DType): Weight matrix element type.
  • ​c_layout (TensorLayout): Layout descriptor for the output tensor.
  • ​a_layout (TensorLayout): Layout descriptor for the activation matrix.
  • ​b_layout (TensorLayout): Layout descriptor for the weight matrix.
  • ​c_engine (TensorEngine): Engine of the output tensor.
  • ​a_engine (TensorEngine): Engine of the activation matrix.
  • ​b_engine (TensorEngine): Engine of the weight matrix.
  • ​ComputeFnType (ElementwiseComputeFn): Type of compute_fn.
  • ​simd_width (Int): Number of elements per vectorized load.
  • ​tile_m (Int): Number of output rows each thread accumulates.
  • ​tile_n (Int): Number of weight rows each thread accumulates.
  • ​num_threads (Int): Threads per block.
  • ​unroll_factor (Int): K-loop unroll factor.
  • ​weight_non_temporal (Bool): Whether to stream the weight matrix.
  • ​accum_type (DType): Accumulation precision type.
  • ​check_bounds_m (Bool): Whether to guard M-tail rows.
  • ​check_bounds_n (Bool): Whether to guard N-tail columns.
  • ​pdl_level (PDLLevel): Programmatic dependent launch level.
  • ​small_m_static_k (Bool): Stage static-K fragments before accumulation.

Args:

Was this page helpful?