IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ihc_collapse_rms_norm_block

def ihc_collapse_rms_norm_block[dtype: DType, hc_mult: Int, num_mix: Int, num_splits: Int, hidden: Int, block_size: Int, poison: Int = Int(0)](output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], gates: TileTensor[.float32, Engine=gates.Engine, address_space=gates.address_space, linear_idx_type=gates.linear_idx_type], streams: TileTensor[dtype, Engine=streams.Engine, address_space=streams.address_space, linear_idx_type=streams.linear_idx_type], partials: TileTensor[.float32, Engine=partials.Engine, address_space=partials.address_space, linear_idx_type=partials.linear_idx_type], base: TileTensor[.float32, Engine=base.Engine, address_space=base.address_space, linear_idx_type=base.linear_idx_type], scale: TileTensor[.float32, Engine=scale.Engine, address_space=scale.address_space, linear_idx_type=scale.linear_idx_type], gamma: TileTensor[dtype, Engine=gamma.Engine, address_space=gamma.address_space, linear_idx_type=gamma.linear_idx_type], hc_magnitude: Float32, hc_eps: Float32, streams_eps: Float32, norm_eps: Float32, row: Int, tid: Int)

Collapses one token's streams and RMS-normalizes the result.

One CTA's work: tid is the thread in a block of block_size that takes token row. The gates come from partials alone, so the streams are read once, straight into the collapse.

Parameters:

  • ​dtype (DType): The stream, output and gamma dtype.
  • ​hc_mult (Int): Number of residual streams.
  • ​num_mix (Int): Gate logits per token: hc_mult (pre only) or 2 * hc_mult (pre then post).
  • ​num_splits (Int): Leading dimension of partials.
  • ​hidden (Int): Width of one stream.
  • ​block_size (Int): Threads per block.
  • ​poison (Int): A planted fault (IHC_POISONS), 0 for none.

Args:

Was this page helpful?