IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Mamba2SSDChunkScanVarlenFwdInplace

struct Mamba2SSDChunkScanVarlenFwdInplace[dt_softplus: Bool = True]

Varlen Mamba-2 SSD chunked scan with an in-place SSM state pool.

Matches mamba_chunk_scan_combined semantics for the Nemotron-H NemotronHMamba2Mixer: per-head scalar A, grouped B/C, per-head dt + dt_bias softplus, and a state reset at each query_start_loc boundary. Gating z and MambaRMSNormGated are applied outside this op.

Each sequence starts from ssm_pool[cache_indices[b]] when has_initial_state[b] is set, and its final state is written back to the same slot, so the graph never round-trips the pool. The kernel math lives in state_space.mamba2_ssd_scan (token-serial scan, used for decode, verify and short launches) and state_space.mamba2_ssd_chunked (chunked state-space-duality scan on tensor cores). On NVIDIA GPUs with bf16 and dstate 128, a launch with at least 512 tokens and an average sequence of at least 128 tokens takes the chunked path.

Tensor shapes: - y: (total_len, nheads, head_dim), output (dtype) [OUT] - x: (total_len, nheads, head_dim), input (dtype) - dt: (total_len, nheads) - A: (nheads,) - B: (total_len, ngroups, dstate) - C: (total_len, ngroups, dstate) - D: (nheads,) optional/empty - dt_bias: (nheads,) optional/empty - ssm_pool: (max_slots, nheads, head_dim, dstate) fp32, or bf16 on Apple GPUs (storage dtype only; the scan accumulates in fp32) [MUT] - query_start_loc: (batch + 1,) int32 - has_initial_state: (batch,) bool optional/empty - cache_indices: (batch,) or (1, batch) uint32, slot indices into ssm_pool - update_state: (batch,) bool optional/empty. A clear flag leaves that sequence's ssm_pool slot untouched (a read-only scan); empty writes every sequence's final state. - xbc: (total_len, nheads * head_dim + 2 * ngroups * dstate) or empty (of any dtype): the conv output that x, B and C are column slices of, in that order. With it, a long prefill on B200 reads their rows by TMA (state_space.mamba2_ssd_chunked).

Parameters​

  • ​dt_softplus (Bool): If True (default), apply softplus to dt + dt_bias.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute(y: T, x: T, dt: T, A: T, B: T, C: T, D: T, dt_bias: T, ssm_pool: T, query_start_loc: T, has_initial_state: T, cache_indices: T, update_state: T, xbc: T, ctx: DeviceContext)

Was this page helpful?