For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mhc_pre_partials_kernel
def mhc_pre_partials_kernel[dtype: DType, fn_dtype: DType, StreamsLayoutType: TensorLayout, FnLayoutType: TensorLayout, PartialLayoutType: TensorLayout, hc_mult: Int, hidden_size: Int, fn_terms: Int, k_splits: Int](partial: TileTensor[.float32, PartialLayoutType, MutAnyOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin], hc_fn: TileTensor[fn_dtype, FnLayoutType, ImmutAnyOrigin], num_rows: Int32)
Projects 16 tokens over one K slice into fp32 partial sums.
Block (mb, ks) owns tokens [16 * mb, 16 * mb + 16) and the contiguous
slice [ks * K / k_splits, (ks + 1) * K / k_splits) of the flat
hc_mult * hidden_size axis, which its waves split into 64-wide steps as
mhc_pre_kernel's waves split K. It stores the 24 projections and the
sum of squares of each token's slice at partial[t, ks * 25 + c], so a
token's partials are one contiguous run for mhc_pre_finish_kernel.
Parameters:
- dtype (
DType): Element type of the streams. - fn_dtype (
DType): Element type of the projection weights. - StreamsLayoutType (
TensorLayout): Layout ofstreams. - FnLayoutType (
TensorLayout): Layout ofhc_fn. - PartialLayoutType (
TensorLayout): Layout ofpartial. - hc_mult (
Int): Number of parallel residual streams. - hidden_size (
Int): Width of one stream. - fn_terms (
Int): Number ofmhc_fn_splitbf16 terms each fp32 weight contributes. - k_splits (
Int): Number of K slices, one per block alongy.
Args:
- partial (
TileTensor[.float32, PartialLayoutType, MutAnyOrigin]): Per-slice sums. Shape:[16 * ceil(T / 16), >= k_splits * (2 * hc_mult + hc_mult**2 + 1)]. - streams (
TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin]): Residual streams. Shape:[T, hc_mult, hidden_size]. - hc_fn (
TileTensor[fn_dtype, FnLayoutType, ImmutAnyOrigin]): Projection weights. Shape:[2 * hc_mult + hc_mult**2, hc_mult * hidden_size]. - num_rows (
Int32): Number of tokensT.