For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
ihc_collapse_rms_norm_kernel
def ihc_collapse_rms_norm_kernel[dtype: DType, OutputLayoutType: TensorLayout, GatesLayoutType: TensorLayout, StreamsLayoutType: TensorLayout, PartialsLayoutType: TensorLayout, BaseLayoutType: TensorLayout, ScaleLayoutType: TensorLayout, GammaLayoutType: TensorLayout, hc_mult: Int, num_mix: Int, num_splits: Int, hidden: Int, block_size: Int, poison: Int](output: TileTensor[dtype, OutputLayoutType, MutUntrackedOrigin], gates: TileTensor[.float32, GatesLayoutType, MutUntrackedOrigin], streams: TileTensor[dtype, StreamsLayoutType, ImmUntrackedOrigin], partials: TileTensor[.float32, PartialsLayoutType, ImmUntrackedOrigin], base: TileTensor[.float32, BaseLayoutType, ImmUntrackedOrigin], scale: TileTensor[.float32, ScaleLayoutType, ImmUntrackedOrigin], gamma: TileTensor[dtype, GammaLayoutType, ImmUntrackedOrigin], hc_magnitude: Float32, hc_eps: Float32, streams_eps: Float32, norm_eps: Float32)
Runs ihc_collapse_rms_norm_block for token block_idx.x.
Parameters:
- dtype (
DType): The stream, output andgammadtype. - OutputLayoutType (
TensorLayout): Layout ofoutput. - GatesLayoutType (
TensorLayout): Layout ofgates. - StreamsLayoutType (
TensorLayout): Layout ofstreams. - PartialsLayoutType (
TensorLayout): Layout ofpartials. - BaseLayoutType (
TensorLayout): Layout ofbase. - ScaleLayoutType (
TensorLayout): Layout ofscale. - GammaLayoutType (
TensorLayout): Layout ofgamma. - hc_mult (
Int): Number of residual streams. - num_mix (
Int): Gate logits per token. - num_splits (
Int): Leading dimension ofpartials. - hidden (
Int): Width of one stream. - block_size (
Int): Threads per block. - poison (
Int): A planted fault (IHC_POISONS), 0 for none.
Args:
- output (
TileTensor[dtype, OutputLayoutType, MutUntrackedOrigin]): Shape:[num_rows, hidden]. - gates (
TileTensor[.float32, GatesLayoutType, MutUntrackedOrigin]): Shape:[num_rows, num_mix]. - streams (
TileTensor[dtype, StreamsLayoutType, ImmUntrackedOrigin]): Shape:[num_rows, hc_mult, hidden]. - partials (
TileTensor[.float32, PartialsLayoutType, ImmUntrackedOrigin]): Shape:[num_splits, num_rows, num_mix + 1]. - base (
TileTensor[.float32, BaseLayoutType, ImmUntrackedOrigin]): Shape:[num_mix]. - scale (
TileTensor[.float32, ScaleLayoutType, ImmUntrackedOrigin]): Shape:[num_mix // hc_mult]. - gamma (
TileTensor[dtype, GammaLayoutType, ImmUntrackedOrigin]): Shape:[hidden]. - hc_magnitude (
Float32): Upper bound ofpost. - hc_eps (
Float32): Added to every gate. - streams_eps (
Float32): Epsilon of the streams' RMS. - norm_eps (
Float32): Epsilon of the output RMSNorm.