IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ssd_state_passing

def ssd_state_passing[kernel_dtype: DType, state_dtype: DType, SLT: TensorLayout, SEngine: TensorEngine, pool_LT: TensorLayout, has_initial_state_LT: TensorLayout, update_state_LT: TensorLayout, Engine: TensorEngine, SlotFn: def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width] & RegisterPassable & ImplicitlyCopyable, DSTATE: Int, VPT: Int = Int(1), entry_bf16: Bool = False, ROW_RULE: Bool = False](slots_i: Int32, batch_i: Int32, nheads_i: Int32, head_dim_i: Int32, info: TileTensor[.int32, SLT, MutUntrackedOrigin, Engine=SEngine], cs: TileTensor[.float32, SLT, MutUntrackedOrigin, Engine=SEngine], states: TileTensor[.float32, SLT, MutUntrackedOrigin, Engine=SEngine], ssm_pool: TileTensor[state_dtype, pool_LT, MutUntrackedOrigin, Engine=Engine], has_initial_state: TileTensor[.bool, has_initial_state_LT, ImmUntrackedOrigin, Engine=Engine], update_state: TileTensor[.bool, update_state_LT, ImmUntrackedOrigin, Engine=Engine], entry_addr: Int64, slot_fn: SlotFn)

Passes the state across the chunks of each sequence.

Replaces states[c] with the state entering chunk c and writes the final state to the pool slot. Grid (ceil(head_dim * DSTATE / F32_VEC / NUM_THREADS / VPT), nheads, batch), block NUM_THREADS; a thread owns VPT runs of four contiguous n, NUM_THREADS * F32_VEC apart.

The pass is bound by DRAM once the states outgrow L2, so the block reads the sequence's chunk decays into shared memory once, and each thread has the next PASS_U chunks' loads in flight before the dependent chain uses them. VPT 4 gives each thread more independent loads when batch x heads fills the GPU; 1 keeps more blocks for a few long sequences.

Was this page helpful?