For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
ihc_mix_gemv_kernel
def ihc_mix_gemv_kernel[dtype: DType, PartialsLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, WeightLayoutType: TensorLayout, num_mix: Int, k: Int, num_warps: Int, poison: Int](partials: TileTensor[.float32, PartialsLayoutType, MutUntrackedOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmUntrackedOrigin], weight: TileTensor[.float32, WeightLayoutType, ImmUntrackedOrigin], num_rows: Int32)
Runs ihc_mix_gemv_block for block (row_block, split).
Parameters:
- dtype (
DType): The stream dtype. - PartialsLayoutType (
TensorLayout): Layout ofpartials. - StreamsLayoutType (
TensorLayout): Layout ofstreams. - WeightLayoutType (
TensorLayout): Layout ofweight. - num_mix (
Int): Number ofweightrows. - k (
Int): Width of the flattened streams,hc_mult * hidden. - num_warps (
Int): Warps per block. - poison (
Int): A planted fault (IHC_POISONS), 0 for none.
Args:
- partials (
TileTensor[.float32, PartialsLayoutType, MutUntrackedOrigin]): Shape:[k // (num_warps * _GEMV_WARP_K), num_rows, num_mix + 1]. - streams (
TileTensor[dtype, StreamsLayoutType, ImmUntrackedOrigin]): The flattened streams. Shape:[num_rows, k]. - weight (
TileTensor[.float32, WeightLayoutType, ImmUntrackedOrigin]): The projection weight. Shape:[num_mix, k]. - num_rows (
Int32): Number of rows.