IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

mamba2_ssd_scan

Mamba-2 SSD (state-space duality) varlen scan with an in-place state pool.

This is the varlen (ragged, query_start_loc) Mamba-2 SSD forward op, matching mamba_chunk_scan_combined semantics for the NemotronHMamba2Mixer (Nemotron-H). It serves both prefill and decode. It differs from the Mamba-1 per-channel A (dim, dstate) diagonal in three ways that the math requires:

  • A is a per-head SCALAR (nheads,) (shared across all head_dim channels and all dstate), not a per-channel (dim, dstate) diagonal.
  • B/C are GROUPED (total_len, ngroups, dstate); nheads/ngroups heads share each group (group_id = h // (nheads // ngroups)).
  • dt is per-head (total_len, nheads) + per-head dt_bias (nheads,), broadcast across head_dim; softplus is applied to dt + dt_bias.

The SSD chunked scan is a parallelism reformulation of the linear recurrence below; in fp32 they are numerically equivalent. The kernels carry the state sequentially per (head, head_dim) channel.

Per-token recurrence (per head h, head_dim channel p, group g):

dt_t     = softplus(dt[t, h] + dt_bias[h])            # scalar per (t, h)
dA_t     = exp(A[h] * dt_t)                           # scalar per (t, h)
state_n  = state_n * dA_t + dt_t * B[t, g, n] * x[t, h, p]   # vector over n
y[t,h,p] = sum_n C[t, g, n] * state_n  +  D[h] * x[t, h, p]

Each sequence starts from zero, or from its pool slot when has_initial_state[b], and its final state is written back to the same slot. D, dt_bias and has_initial_state may be empty to drop that term. The dstate axis of ssm_pool must be contiguous.

update_state[b] (empty means all true) says whether sequence b's final state is written back. With it clear the scan only reads the pool, which lets a speculative verify run on the live state and leave it as it was; under ROW_RULE a prefill row writes its commit point anyway.

x, dt, B, C and the slot indices are read through operand functions, so a graph op can fold a slice of a larger tensor into its loads instead of copying it first.

comptime values​

LOG2E​

comptime LOG2E = 1.4426950408889634

Functions​

Was this page helpful?