IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ihc_collapse_rms_norm_kernel

def ihc_collapse_rms_norm_kernel[dtype: DType, OutputLayoutType: TensorLayout, GatesLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, PartialsLayoutType: TensorLayout, BaseLayoutType: TensorLayout, ScaleLayoutType: TensorLayout, GammaLayoutType: TensorLayout, hc_mult: Int, num_mix: Int, num_splits: Int, hidden: Int, block_size: Int, poison: Int](output: TileTensor[dtype, OutputLayoutType, MutUntrackedOrigin], gates: TileTensor[.float32, GatesLayoutType, MutUntrackedOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmUntrackedOrigin], partials: TileTensor[.float32, PartialsLayoutType, ImmUntrackedOrigin], base: TileTensor[.float32, BaseLayoutType, ImmUntrackedOrigin], scale: TileTensor[.float32, ScaleLayoutType, ImmUntrackedOrigin], gamma: TileTensor[dtype, GammaLayoutType, ImmUntrackedOrigin], hc_magnitude: Float32, hc_eps: Float32, streams_eps: Float32, norm_eps: Float32)

Runs ihc_collapse_rms_norm_block for token block_idx.x.

Parameters:

  • ​dtype (DType): The stream, output and gamma dtype.
  • ​OutputLayoutType (TensorLayout): Layout of output.
  • ​GatesLayoutType (TensorLayout): Layout of gates.
  • ​StreamsLayoutType (TensorLayout): Layout of streams.
  • ​PartialsLayoutType (TensorLayout): Layout of partials.
  • ​BaseLayoutType (TensorLayout): Layout of base.
  • ​ScaleLayoutType (TensorLayout): Layout of scale.
  • ​GammaLayoutType (TensorLayout): Layout of gamma.
  • ​hc_mult (Int): Number of residual streams.
  • ​num_mix (Int): Gate logits per token.
  • ​num_splits (Int): Leading dimension of partials.
  • ​hidden (Int): Width of one stream.
  • ​block_size (Int): Threads per block.
  • ​poison (Int): A planted fault (IHC_POISONS), 0 for none.

Args:

Was this page helpful?