IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mhc_pre

def mhc_pre[hc_mult: Int, hidden_size: Int, hc_sinkhorn_iters: Int, has_norm: Bool, fn_terms: Int = Int(3)](post: TileTensor[.float32, Engine=post.Engine, address_space=post.address_space, linear_idx_type=post.linear_idx_type], comb: TileTensor[.float32, Engine=comb.Engine, address_space=comb.address_space, linear_idx_type=comb.linear_idx_type], collapsed: TileTensor[Engine=collapsed.Engine, address_space=collapsed.address_space, linear_idx_type=collapsed.linear_idx_type], pre: TileTensor[.float32, Engine=pre.Engine, address_space=pre.address_space, linear_idx_type=pre.linear_idx_type], streams: TileTensor[collapsed.dtype, Engine=streams.Engine, address_space=streams.address_space, linear_idx_type=streams.linear_idx_type], hc_fn: TileTensor[Engine=hc_fn.Engine, address_space=hc_fn.address_space, linear_idx_type=hc_fn.linear_idx_type], base: TileTensor[.float32, Engine=base.Engine, address_space=base.address_space, linear_idx_type=base.linear_idx_type], scale: TileTensor[.float32, Engine=scale.Engine, address_space=scale.address_space, linear_idx_type=scale.linear_idx_type], gamma: TileTensor[collapsed.dtype, Engine=gamma.Engine, address_space=gamma.address_space, linear_idx_type=gamma.linear_idx_type], rms_eps: Float32, hc_eps: Float32, norm_eps: Float32, context: DeviceContext, force_k_splits: Int = Int(0))

Launches the fused mHC pre site on blocks of 16 tokens and 256 threads.

Computes, per token t with x = streams[t] flattened to hc_mult * hidden_size values:

  • mix = (x @ hc_fn.T) * rsqrt(mean(x * x) + rms_eps);
  • pre, post and comb from mix, as hyper_connection_gates does;
  • collapsed[t] = sum_j pre[t, j] * streams[t, j], rounded to the streams' dtype, then with has_norm RMS-normalized by gamma.

The projection runs on bf16 MFMA with fp32 accumulation. A bf16 hc_fn is one exact term; an fp32 hc_fn is split into fn_terms bf16 terms, and at 3 their products are exact, so the result is at least as accurate as an fp32 GEMM. Requires gfx950.

mhc_pre_launch_config picks the K-splits KS that bring the grid to about 256 blocks, and KS = 4 from 129 to 192 token blocks, where the fused grid would leave CUs idle. At KS = 1 one kernel does the whole site. Above it, KS blocks per 16 tokens each project a slice of K into a per-call fp32 workspace, and a second launch on the same stream sums each token's slices in a fixed order and finishes it, one block per token. Both forms are deterministic.

Parameters:

  • ​hc_mult (Int): Number of parallel residual streams. Only 4.
  • ​hidden_size (Int): Width of one stream: a multiple of 1024 in [1024, 8192].
  • ​hc_sinkhorn_iters (Int): Sinkhorn-Knopp iterations used to project comb.
  • ​has_norm (Bool): Whether to RMS-normalize the collapsed row by gamma.
  • ​fn_terms (Int): Test-only: number of bf16 terms per fp32 hc_fn weight, 1 to 3. Callers keep the default 3; fewer drop precision and exist so tests can check that every term matters. Ignored for a bf16 hc_fn.

Args:

Raises:

If force_k_splits is not one mhc_pre_launch_config accepts, a shape disagrees with hc_mult, hidden_size or the token count of streams, or a tensor is not laid out for the kernel's 16-byte vector accesses.

Was this page helpful?