For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

selective_scan_update_cpu

def selective_scan_update_cpu[kernel_dtype: DType, DSTATE: Int](batch: Int, dim: Int, group_size: Int, delta_softplus: Int8, state_out: TileTensor[kernel_dtype, Storage=state_out.Storage, address_space=state_out.address_space, linear_idx_type=state_out.linear_idx_type], output: TileTensor[kernel_dtype, Storage=output.Storage, address_space=output.address_space, linear_idx_type=output.linear_idx_type], state_in: TileTensor[kernel_dtype, Storage=state_in.Storage, address_space=state_in.address_space, linear_idx_type=state_in.linear_idx_type], x: TileTensor[kernel_dtype, Storage=x.Storage, address_space=x.address_space, linear_idx_type=x.linear_idx_type], dt: TileTensor[kernel_dtype, Storage=dt.Storage, address_space=dt.address_space, linear_idx_type=dt.linear_idx_type], A: TileTensor[kernel_dtype, Storage=A.Storage, address_space=A.address_space, linear_idx_type=A.linear_idx_type], B: TileTensor[kernel_dtype, Storage=B.Storage, address_space=B.address_space, linear_idx_type=B.linear_idx_type], C: TileTensor[kernel_dtype, Storage=C.Storage, address_space=C.address_space, linear_idx_type=C.linear_idx_type], D: TileTensor[kernel_dtype, Storage=D.Storage, address_space=D.address_space, linear_idx_type=D.linear_idx_type], z: TileTensor[kernel_dtype, Storage=z.Storage, address_space=z.address_space, linear_idx_type=z.linear_idx_type], dt_bias: TileTensor[kernel_dtype, Storage=dt_bias.Storage, address_space=dt_bias.address_space, linear_idx_type=dt_bias.linear_idx_type], state_out_strides: IndexList[Int(3)], output_strides: IndexList[Int(2)], state_in_strides: IndexList[Int(3)], x_strides: IndexList[Int(2)], dt_strides: IndexList[Int(2)], A_strides: IndexList[Int(2)], B_strides: IndexList[Int(3)], C_strides: IndexList[Int(3)], D_strides: IndexList[Int(1)], z_strides: IndexList[Int(2)], dt_bias_strides: IndexList[Int(1)], ctx: Optional[DeviceContext] = None)

CPU kernel for selective scan update (single step).

Each worker processes one (batch, dim) pair and advances the SSM recurrence by a single autoregressive timestep, reading the previous state from state_in and writing the updated state to state_out and the step output to output.

Parameters:

kernel_dtype (DType): Element type of the input and output tensors.
DSTATE (Int): Number of SSM state elements per (batch, dim) pair.

Args:

batch (Int): Number of sequences processed in parallel.
dim (Int): Hidden dimension, equal to the number of channels per sequence position.
group_size (Int): Number of dims per group; dims in the same group share B and C inputs.
delta_softplus (Int8): Nonzero applies softplus to dt before the scan recurrence.
state_out (TileTensor[kernel_dtype, Storage=state_out.Storage, address_space=state_out.address_space, linear_idx_type=state_out.linear_idx_type]): Updated SSM state tensor of shape (batch, dim, DSTATE), written.
output (TileTensor[kernel_dtype, Storage=output.Storage, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): Step output tensor of shape (batch, dim), written.
state_in (TileTensor[kernel_dtype, Storage=state_in.Storage, address_space=state_in.address_space, linear_idx_type=state_in.linear_idx_type]): Previous SSM state tensor of shape (batch, dim, DSTATE), read.
x (TileTensor[kernel_dtype, Storage=x.Storage, address_space=x.address_space, linear_idx_type=x.linear_idx_type]): Input value tensor of shape (batch, dim) for the current timestep, read.
dt (TileTensor[kernel_dtype, Storage=dt.Storage, address_space=dt.address_space, linear_idx_type=dt.linear_idx_type]): Time-step tensor of shape (batch, dim), read.
A (TileTensor[kernel_dtype, Storage=A.Storage, address_space=A.address_space, linear_idx_type=A.linear_idx_type]): SSM recurrence matrix of shape (dim, DSTATE), read.
B (TileTensor[kernel_dtype, Storage=B.Storage, address_space=B.address_space, linear_idx_type=B.linear_idx_type]): SSM input projection of shape (batch, n_groups, DSTATE), read.
C (TileTensor[kernel_dtype, Storage=C.Storage, address_space=C.address_space, linear_idx_type=C.linear_idx_type]): SSM output projection of shape (batch, n_groups, DSTATE), read.
D (TileTensor[kernel_dtype, Storage=D.Storage, address_space=D.address_space, linear_idx_type=D.linear_idx_type]): Skip connection vector of shape (dim,), read; added as D * x to the output when present.
z (TileTensor[kernel_dtype, Storage=z.Storage, address_space=z.address_space, linear_idx_type=z.linear_idx_type]): Gating tensor of shape (batch, dim), read; gates the output via z * sigmoid(z) when present.
dt_bias (TileTensor[kernel_dtype, Storage=dt_bias.Storage, address_space=dt_bias.address_space, linear_idx_type=dt_bias.linear_idx_type]): Bias vector of shape (dim,), read; added to dt before softplus when present.
state_out_strides (IndexList[Int(3)]): 3D strides (batch, dim, DSTATE) for indexing state_out.
output_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing output.
state_in_strides (IndexList[Int(3)]): 3D strides (batch, dim, DSTATE) for indexing state_in.
x_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing x.
dt_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing dt.
A_strides (IndexList[Int(2)]): 2D strides (dim, DSTATE) for indexing A.
B_strides (IndexList[Int(3)]): 3D strides (batch, n_groups, DSTATE) for indexing B.
C_strides (IndexList[Int(3)]): 3D strides (batch, n_groups, DSTATE) for indexing C.
D_strides (IndexList[Int(1)]): 1D strides (dim,) for indexing D.
z_strides (IndexList[Int(2)]): 2D strides (batch, dim) for indexing z.
dt_bias_strides (IndexList[Int(1)]): 1D strides (dim,) for indexing dt_bias.
ctx (Optional[DeviceContext]): Device context used to drive the parallel worker loop (defaults to None).