For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
ssd_state_passing
def ssd_state_passing[kernel_dtype: DType, state_dtype: DType, SLT: TensorLayout, SEngine: TensorEngine, pool_LT: TensorLayout, has_initial_state_LT: TensorLayout, update_state_LT: TensorLayout, Engine: TensorEngine, SlotFn: def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width] & RegisterPassable & ImplicitlyCopyable, DSTATE: Int, VPT: Int = Int(1), entry_bf16: Bool = False, ROW_RULE: Bool = False](slots_i: Int32, batch_i: Int32, nheads_i: Int32, head_dim_i: Int32, info: TileTensor[.int32, SLT, MutUntrackedOrigin, Engine=SEngine], cs: TileTensor[.float32, SLT, MutUntrackedOrigin, Engine=SEngine], states: TileTensor[.float32, SLT, MutUntrackedOrigin, Engine=SEngine], ssm_pool: TileTensor[state_dtype, pool_LT, MutUntrackedOrigin, Engine=Engine], has_initial_state: TileTensor[.bool, has_initial_state_LT, ImmUntrackedOrigin, Engine=Engine], update_state: TileTensor[.bool, update_state_LT, ImmUntrackedOrigin, Engine=Engine], entry_addr: Int64, slot_fn: SlotFn)
Passes the state across the chunks of each sequence.
Replaces states[c] with the state entering chunk c and writes the final
state to the pool slot. Grid (ceil(head_dim * DSTATE / F32_VEC / NUM_THREADS / VPT), nheads, batch), block NUM_THREADS; a thread owns
VPT runs of four contiguous n, NUM_THREADS * F32_VEC apart.
The pass is bound by DRAM once the states outgrow L2, so the block reads
the sequence's chunk decays into shared memory once, and each thread has
the next PASS_U chunks' loads in flight before the dependent chain uses
them. VPT 4 gives each thread more independent loads when batch x heads
fills the GPU; 1 keeps more blocks for a few long sequences.