IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Mamba2SSDChunkScanVarlenFwdInplace

struct Mamba2SSDChunkScanVarlenFwdInplace[dt_softplus: Bool = True]

Varlen Mamba-2 SSD chunked scan with an in-place SSM state pool.

Matches mamba_chunk_scan_combined semantics for the Nemotron-H NemotronHMamba2Mixer: per-head scalar A, grouped B/C, per-head dt + dt_bias softplus, and a state reset at each query_start_loc boundary. Gating z and MambaRMSNormGated are applied outside this op.

Each sequence starts from ssm_pool[cache_indices[b]] when has_initial_state[b] is set, and its final state is written back to the same slot, so the graph never round-trips the pool. The kernel math lives in state_space.mamba2_ssd_scan.

Tensor shapes: - y: (total_len, nheads, head_dim), output (dtype) [OUT] - x: (total_len, nheads, head_dim), input (dtype) - dt: (total_len, nheads) - A: (nheads,) - B: (total_len, ngroups, dstate) - C: (total_len, ngroups, dstate) - D: (nheads,) optional/empty - dt_bias: (nheads,) optional/empty - ssm_pool: (max_slots, nheads, head_dim, dstate) fp32, or bf16 on Apple GPUs (storage dtype only; the scan accumulates in fp32) [MUT] - query_start_loc: (batch + 1,) int32 - has_initial_state: (batch,) bool optional/empty - cache_indices: (batch,) or (1, batch) uint32, slot indices into ssm_pool - update_state: (batch,) bool optional/empty. A clear flag leaves that sequence's ssm_pool slot untouched (a read-only scan); empty writes every sequence's final state.

Parameters​

  • ​dt_softplus (Bool): If True (default), apply softplus to dt + dt_bias.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[dtype: DType, state_dtype: DType, target: StringSpan[ImmStaticOrigin]](y: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=y.static_spec], x: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=x.static_spec], dt: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=dt.static_spec], A: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=A.static_spec], B: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=B.static_spec], C: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=C.static_spec], D: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=D.static_spec], dt_bias: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=dt_bias.static_spec], ssm_pool: ManagedTensorSlice[IOSpec[_, _].MutableInput, static_spec=ssm_pool.static_spec], query_start_loc: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=query_start_loc.static_spec], has_initial_state: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=has_initial_state.static_spec], cache_indices: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=cache_indices.static_spec], update_state: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=update_state.static_spec], ctx: DeviceContextBase)

Was this page helpful?