For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mhc_pre_kernel
def mhc_pre_kernel[dtype: DType, fn_dtype: DType, PostLayoutType: TensorLayout, CombLayoutType: TensorLayout, CollapsedLayoutType: TensorLayout, PreLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, FnLayoutType: TensorLayout, BaseLayoutType: TensorLayout, ScaleLayoutType: TensorLayout, GammaLayoutType: TensorLayout, hc_mult: Int, hidden_size: Int, hc_sinkhorn_iters: Int, has_norm: Bool, fn_terms: Int](post: TileTensor[.float32, PostLayoutType, MutAnyOrigin], comb: TileTensor[.float32, CombLayoutType, MutAnyOrigin], collapsed: TileTensor[dtype, CollapsedLayoutType, MutAnyOrigin], pre: TileTensor[.float32, PreLayoutType, MutAnyOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin], hc_fn: TileTensor[fn_dtype, FnLayoutType, ImmutAnyOrigin], base: TileTensor[.float32, BaseLayoutType, ImmutAnyOrigin], scale: TileTensor[.float32, ScaleLayoutType, ImmutAnyOrigin], gamma: TileTensor[dtype, GammaLayoutType, ImmutAnyOrigin], rms_eps: Float32, hc_eps: Float32, norm_eps: Float32, num_rows: Int32)
Runs a whole mHC pre site for 16 tokens: projection, gates, collapse.
Wave j projects stream j (a quarter of K = hc_mult * hidden_size)
for the block's 16 tokens on v_mfma_f32_16x16x32_bf16, two 16-wide
tiles covering the 24 mix outputs (see _mhc_pre_project). The waves
then reduce through LDS in a fixed order, and each wave finishes four
tokens: the RMS post-scale, the gates, then the collapse.
Parameters:
- dtype (
DType): Element type of the streams and the collapsed output. - fn_dtype (
DType): Element type of the projection weights. - PostLayoutType (
TensorLayout): Layout ofpost. - CombLayoutType (
TensorLayout): Layout ofcomb. - CollapsedLayoutType (
TensorLayout): Layout ofcollapsed. - PreLayoutType (
TensorLayout): Layout ofpre. - StreamsLayoutType (
TensorLayout): Layout ofstreams. - FnLayoutType (
TensorLayout): Layout ofhc_fn. - BaseLayoutType (
TensorLayout): Layout ofbase. - ScaleLayoutType (
TensorLayout): Layout ofscale. - GammaLayoutType (
TensorLayout): Layout ofgamma. - hc_mult (
Int): Number of parallel residual streams. - hidden_size (
Int): Width of one stream. - hc_sinkhorn_iters (
Int): Sinkhorn-Knopp iterations used to projectcomb. - has_norm (
Bool): Whether to RMS-normalize the collapsed row bygamma. - fn_terms (
Int): Number ofmhc_fn_splitbf16 terms each fp32 weight contributes.
Args:
- post (
TileTensor[.float32, PostLayoutType, MutAnyOrigin]): Sublayer-output placement weights. Shape:[T, hc_mult]. - comb (
TileTensor[.float32, CombLayoutType, MutAnyOrigin]): Stream mixer;comb[t, j, k]weighs input streamjin output streamk. Shape:[T, hc_mult, hc_mult]. - collapsed (
TileTensor[dtype, CollapsedLayoutType, MutAnyOrigin]): The collapsed (optionally normalized) sequence. Shape:[T, hidden_size]. - pre (
TileTensor[.float32, PreLayoutType, MutAnyOrigin]): Stream-collapse weights. Shape:[T, hc_mult]. - streams (
TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin]): Residual streams. Shape:[T, hc_mult, hidden_size]. - hc_fn (
TileTensor[fn_dtype, FnLayoutType, ImmutAnyOrigin]): Projection weights,pre,post,combrows. Shape:[2 * hc_mult + hc_mult**2, hc_mult * hidden_size]. - base (
TileTensor[.float32, BaseLayoutType, ImmutAnyOrigin]): Per-output bias. Shape:[2 * hc_mult + hc_mult**2]. - scale (
TileTensor[.float32, ScaleLayoutType, ImmutAnyOrigin]): Per-output scale, inpre,post,comborder. Shape:[3]. - gamma (
TileTensor[dtype, GammaLayoutType, ImmutAnyOrigin]): RMSNorm weight. Shape:[hidden_size], read only withhas_norm. - rms_eps (
Float32): Epsilon of the RMS scale on the projection. - hc_eps (
Float32): Epsilon guarding the gates and the Sinkhorn divisions. - norm_eps (
Float32): Epsilon of the collapsed-row RMSNorm. - num_rows (
Int32): Number of tokensT.