IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mhc_pre_partials_kernel

def mhc_pre_partials_kernel[dtype: DType, fn_dtype: DType, StreamsLayoutType: TensorLayout, FnLayoutType: TensorLayout, PartialLayoutType: TensorLayout, hc_mult: Int, hidden_size: Int, fn_terms: Int, k_splits: Int](partial: TileTensor[.float32, PartialLayoutType, MutAnyOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin], hc_fn: TileTensor[fn_dtype, FnLayoutType, ImmutAnyOrigin], num_rows: Int32)

Projects 16 tokens over one K slice into fp32 partial sums.

Block (mb, ks) owns tokens [16 * mb, 16 * mb + 16) and the contiguous slice [ks * K / k_splits, (ks + 1) * K / k_splits) of the flat hc_mult * hidden_size axis, which its waves split into 64-wide steps as mhc_pre_kernel's waves split K. It stores the 24 projections and the sum of squares of each token's slice at partial[t, ks * 25 + c], so a token's partials are one contiguous run for mhc_pre_finish_kernel.

Parameters:

  • ​dtype (DType): Element type of the streams.
  • ​fn_dtype (DType): Element type of the projection weights.
  • ​StreamsLayoutType (TensorLayout): Layout of streams.
  • ​FnLayoutType (TensorLayout): Layout of hc_fn.
  • ​PartialLayoutType (TensorLayout): Layout of partial.
  • ​hc_mult (Int): Number of parallel residual streams.
  • ​hidden_size (Int): Width of one stream.
  • ​fn_terms (Int): Number of mhc_fn_split bf16 terms each fp32 weight contributes.
  • ​k_splits (Int): Number of K slices, one per block along y.

Args:

Was this page helpful?