IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ihc_mix_gemv_block

def ihc_mix_gemv_block[dtype: DType, num_mix: Int, num_warps: Int, poison: Int = Int(0)](partials: TileTensor[.float32, Engine=partials.Engine, address_space=partials.address_space, linear_idx_type=partials.linear_idx_type], streams: TileTensor[dtype, Engine=streams.Engine, address_space=streams.address_space, linear_idx_type=streams.linear_idx_type], weight: TileTensor[.float32, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], num_rows: Int, row_block: Int, split: Int, tid: Int)

Projects _GEMV_TILES 16-row tiles onto one K split on tensor cores.

One CTA's work: tid is the thread in a block of num_warps warps that takes rows row_block * 64 ... and split split of the columns. Warp w owns _GEMV_WARP_K columns of the flattened streams and runs m16n8k16 MMAs over them for every row tile. The float32 weight is split into a bfloat16 hi and lo = weight - hi, two MMAs per tile, which keeps about 16 bits of it; the bfloat16 streams are exact.

K is a reduction axis, so it can be permuted freely as long as A and B agree. Lane (g, t) = (lane // 4, lane % 4) loads 8 contiguous columns 8t .. 8t+7 of each 32-column chunk for rows g and g + 8; the first four are its fragment of the chunk's first MMA, the last four of the second, and it loads row g of the weight at the same columns.

Parameters:

  • ​dtype (DType): The stream dtype.
  • ​num_mix (Int): Number of weight rows.
  • ​num_warps (Int): Warps per block.
  • ​poison (Int): A planted fault (IHC_POISONS), 0 for none.

Args:

Was this page helpful?