For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
ihc_collapse_rms_norm
def ihc_collapse_rms_norm[hc_mult: Int, num_mix: Int, num_splits: Int, hidden: Int, poison: Int = Int(0)](output: TileTensor[Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], gates: TileTensor[.float32, Engine=gates.Engine, address_space=gates.address_space, linear_idx_type=gates.linear_idx_type], streams: TileTensor[output.dtype, Engine=streams.Engine, address_space=streams.address_space, linear_idx_type=streams.linear_idx_type], partials: TileTensor[.float32, Engine=partials.Engine, address_space=partials.address_space, linear_idx_type=partials.linear_idx_type], base: TileTensor[.float32, Engine=base.Engine, address_space=base.address_space, linear_idx_type=base.linear_idx_type], scale: TileTensor[.float32, Engine=scale.Engine, address_space=scale.address_space, linear_idx_type=scale.linear_idx_type], gamma: TileTensor[output.dtype, Engine=gamma.Engine, address_space=gamma.address_space, linear_idx_type=gamma.linear_idx_type], hc_magnitude: Float32, hc_eps: Float32, streams_eps: Float32, norm_eps: Float32, context: DeviceContext)
Launches the iHC collapse + RMSNorm kernel, one block per token.
Parameters:
- hc_mult (
Int): Number of residual streams. - num_mix (
Int): Gate logits per token,hc_multor2 * hc_mult. - num_splits (
Int): Leading dimension ofpartials. - hidden (
Int): Width of one stream. - poison (
Int): A planted fault (IHC_POISONS), 0 for none.
Args:
- output (
TileTensor[Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): The normalized collapse. Shape:[num_rows, hidden]. - gates (
TileTensor[.float32, Engine=gates.Engine, address_space=gates.address_space, linear_idx_type=gates.linear_idx_type]):pre, thenpostifpartialscarries it. Shape:[num_rows, num_mix]. - streams (
TileTensor[output.dtype, Engine=streams.Engine, address_space=streams.address_space, linear_idx_type=streams.linear_idx_type]): Shape:[num_rows, hc_mult, hidden]. - partials (
TileTensor[.float32, Engine=partials.Engine, address_space=partials.address_space, linear_idx_type=partials.linear_idx_type]):ihc_mix_gemv's output. Shape:[num_splits, num_rows, num_mix + 1]. - base (
TileTensor[.float32, Engine=base.Engine, address_space=base.address_space, linear_idx_type=base.linear_idx_type]): Per-logit bias. Shape:[num_mix]. - scale (
TileTensor[.float32, Engine=scale.Engine, address_space=scale.address_space, linear_idx_type=scale.linear_idx_type]): One scale per band. Shape:[num_mix // hc_mult]. - gamma (
TileTensor[output.dtype, Engine=gamma.Engine, address_space=gamma.address_space, linear_idx_type=gamma.linear_idx_type]): The RMSNorm weight. Shape:[hidden]. - hc_magnitude (
Float32): Upper bound ofpost. - hc_eps (
Float32): Added to every gate. - streams_eps (
Float32): Epsilon of the streams' RMS. - norm_eps (
Float32): Epsilon of the output RMSNorm. - context (
DeviceContext): The device context.
Raises:
If a width disagrees with the parameters or a row count with the streams.