IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mhc_pre_kernel

def mhc_pre_kernel[dtype: DType, fn_dtype: DType, PostLayoutType: TensorLayout, CombLayoutType: TensorLayout, CollapsedLayoutType: TensorLayout, PreLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, FnLayoutType: TensorLayout, BaseLayoutType: TensorLayout, ScaleLayoutType: TensorLayout, GammaLayoutType: TensorLayout, hc_mult: Int, hidden_size: Int, hc_sinkhorn_iters: Int, has_norm: Bool, fn_terms: Int](post: TileTensor[.float32, PostLayoutType, MutAnyOrigin], comb: TileTensor[.float32, CombLayoutType, MutAnyOrigin], collapsed: TileTensor[dtype, CollapsedLayoutType, MutAnyOrigin], pre: TileTensor[.float32, PreLayoutType, MutAnyOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmutAnyOrigin], hc_fn: TileTensor[fn_dtype, FnLayoutType, ImmutAnyOrigin], base: TileTensor[.float32, BaseLayoutType, ImmutAnyOrigin], scale: TileTensor[.float32, ScaleLayoutType, ImmutAnyOrigin], gamma: TileTensor[dtype, GammaLayoutType, ImmutAnyOrigin], rms_eps: Float32, hc_eps: Float32, norm_eps: Float32, num_rows: Int32)

Runs a whole mHC pre site for 16 tokens: projection, gates, collapse.

Wave j projects stream j (a quarter of K = hc_mult * hidden_size) for the block's 16 tokens on v_mfma_f32_16x16x32_bf16, two 16-wide tiles covering the 24 mix outputs (see _mhc_pre_project). The waves then reduce through LDS in a fixed order, and each wave finishes four tokens: the RMS post-scale, the gates, then the collapse.

Parameters:

  • ​dtype (DType): Element type of the streams and the collapsed output.
  • ​fn_dtype (DType): Element type of the projection weights.
  • ​PostLayoutType (TensorLayout): Layout of post.
  • ​CombLayoutType (TensorLayout): Layout of comb.
  • ​CollapsedLayoutType (TensorLayout): Layout of collapsed.
  • ​PreLayoutType (TensorLayout): Layout of pre.
  • ​StreamsLayoutType (TensorLayout): Layout of streams.
  • ​FnLayoutType (TensorLayout): Layout of hc_fn.
  • ​BaseLayoutType (TensorLayout): Layout of base.
  • ​ScaleLayoutType (TensorLayout): Layout of scale.
  • ​GammaLayoutType (TensorLayout): Layout of gamma.
  • ​hc_mult (Int): Number of parallel residual streams.
  • ​hidden_size (Int): Width of one stream.
  • ​hc_sinkhorn_iters (Int): Sinkhorn-Knopp iterations used to project comb.
  • ​has_norm (Bool): Whether to RMS-normalize the collapsed row by gamma.
  • ​fn_terms (Int): Number of mhc_fn_split bf16 terms each fp32 weight contributes.

Args:

Was this page helpful?