For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

selective_scan_update_gpu

def selective_scan_update_gpu[kernel_dtype: DType, DSTATE: Int, state_out_LT: TensorLayout, output_LT: TensorLayout, state_in_LT: TensorLayout, x_LT: TensorLayout, dt_LT: TensorLayout, A_LT: TensorLayout, B_LT: TensorLayout, C_LT: TensorLayout, D_LT: TensorLayout, z_LT: TensorLayout, dt_bias_LT: TensorLayout](total_batch_dim: Int32, batch: Int32, dim: Int32, group_size: Int32, delta_softplus: Int8, state_out: TileTensor[kernel_dtype, state_out_LT, MutUntrackedOrigin], output: TileTensor[kernel_dtype, output_LT, MutUntrackedOrigin], state_in: TileTensor[kernel_dtype, state_in_LT, MutUntrackedOrigin], x: TileTensor[kernel_dtype, x_LT, MutUntrackedOrigin], dt: TileTensor[kernel_dtype, dt_LT, MutUntrackedOrigin], A: TileTensor[kernel_dtype, A_LT, MutUntrackedOrigin], B: TileTensor[kernel_dtype, B_LT, MutUntrackedOrigin], C: TileTensor[kernel_dtype, C_LT, MutUntrackedOrigin], D: TileTensor[kernel_dtype, D_LT, MutUntrackedOrigin], z: TileTensor[kernel_dtype, z_LT, MutUntrackedOrigin], dt_bias: TileTensor[kernel_dtype, dt_bias_LT, MutUntrackedOrigin], state_out_strides: IndexList[Int(3)], output_strides: IndexList[Int(2)], state_in_strides: IndexList[Int(3)], x_strides: IndexList[Int(2)], dt_strides: IndexList[Int(2)], A_strides: IndexList[Int(2)], B_strides: IndexList[Int(3)], C_strides: IndexList[Int(3)], D_strides: IndexList[Int(1)], z_strides: IndexList[Int(2)], dt_bias_strides: IndexList[Int(1)])

GPU kernel for selective scan update (single step).

Each thread processes one (batch, dim) pair. Reads initial state from state_in, writes updated state to state_out.

Parameters:

• ​kernel_dtype (DType): Element type of the input and output tensors.
• ​DSTATE (Int): Number of SSM state elements per (batch, dim) pair.
• ​state_out_LT (TensorLayout): Memory layout of the state_out tensor.
• ​output_LT (TensorLayout): Memory layout of the output tensor.
• ​state_in_LT (TensorLayout): Memory layout of the state_in tensor.
• ​x_LT (TensorLayout): Memory layout of the x input tensor.
• ​dt_LT (TensorLayout): Memory layout of the dt time-step tensor.
• ​A_LT (TensorLayout): Memory layout of the A recurrence matrix.
• ​B_LT (TensorLayout): Memory layout of the B input projection tensor.
• ​C_LT (TensorLayout): Memory layout of the C output projection tensor.
• ​D_LT (TensorLayout): Memory layout of the D skip connection tensor.
• ​z_LT (TensorLayout): Memory layout of the z gating tensor.
• ​dt_bias_LT (TensorLayout): Memory layout of the dt_bias tensor.

Args:

• ​total_batch_dim (Int32): Total number of (batch, dim) pairs launched, equal to batch * dim, used for thread bounds checking.
• ​batch (Int32): Number of sequences processed in parallel.
• ​dim (Int32): Hidden dimension, equal to the number of channels per sequence position.
• ​group_size (Int32): Number of dims per group; dims in the same group share B and C inputs.
• ​delta_softplus (Int8): Nonzero applies softplus to dt before the scan recurrence.
• ​state_out (TileTensor[kernel_dtype, state_out_LT, MutUntrackedOrigin]): Updated SSM state tensor of shape (batch, dim, DSTATE), written.
• ​output (TileTensor[kernel_dtype, output_LT, MutUntrackedOrigin]): Step output tensor of shape (batch, dim), written.
• ​state_in (TileTensor[kernel_dtype, state_in_LT, MutUntrackedOrigin]): Previous SSM state tensor of shape (batch, dim, DSTATE), read.
• ​x (TileTensor[kernel_dtype, x_LT, MutUntrackedOrigin]): Input value tensor of shape (batch, dim) for the current timestep, read.
• ​dt (TileTensor[kernel_dtype, dt_LT, MutUntrackedOrigin]): Time-step tensor of shape (batch, dim), read.
• ​A (TileTensor[kernel_dtype, A_LT, MutUntrackedOrigin]): SSM recurrence matrix of shape (dim, DSTATE), read.
• ​B (TileTensor[kernel_dtype, B_LT, MutUntrackedOrigin]): SSM input projection of shape (batch, n_groups, DSTATE), read.
• ​C (TileTensor[kernel_dtype, C_LT, MutUntrackedOrigin]): SSM output projection of shape (batch, n_groups, DSTATE), read.
• ​D (TileTensor[kernel_dtype, D_LT, MutUntrackedOrigin]): Skip connection vector of shape (dim,), read; added as D * x to the output when present.
• ​z (TileTensor[kernel_dtype, z_LT, MutUntrackedOrigin]): Gating tensor of shape (batch, dim), read; gates the output via z * sigmoid(z) when present.
• ​dt_bias (TileTensor[kernel_dtype, dt_bias_LT, MutUntrackedOrigin]): Bias vector of shape (dim,), read; added to dt before softplus when present.
• ​state_out_strides (IndexList[Int(3)]): 3D strides (batch, dim, DSTATE) for indexing state_out.
• ​output_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing output.
• ​state_in_strides (IndexList[Int(3)]): 3D strides (batch, dim, DSTATE) for indexing state_in.
• ​x_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing x.
• ​dt_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing dt.
• ​A_strides (IndexList[Int(2)]): 2D strides (dim, DSTATE) for indexing A.
• ​B_strides (IndexList[Int(3)]): 3D strides (batch, n_groups, DSTATE) for indexing B.
• ​C_strides (IndexList[Int(3)]): 3D strides (batch, n_groups, DSTATE) for indexing C.
• ​D_strides (IndexList[Int(1)]): 1D strides (dim,) for indexing D.
• ​z_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing z.
• ​dt_bias_strides (IndexList[Int(1)]): 1D strides (dim,) for indexing dt_bias.