For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ssd_combined_cpu

def ssd_combined_cpu[kernel_dtype: DType, DSTATE: Int, output_LT: TensorLayout, x_LT: TensorLayout, out_z_LT: TensorLayout, residual_LT: TensorLayout, u_LT: TensorLayout, delta_LT: TensorLayout, A_LT: TensorLayout, B_LT: TensorLayout, C_LT: TensorLayout, D_LT: TensorLayout, z_LT: TensorLayout, delta_bias_LT: TensorLayout, gamma_LT: TensorLayout](batch: Int, dim: Int, seqlen: Int, group_size: Int, delta_softplus: Int8, output: TileTensor[kernel_dtype, output_LT, MutAnyOrigin], x: TileTensor[kernel_dtype, x_LT, MutAnyOrigin], out_z: TileTensor[kernel_dtype, out_z_LT, MutAnyOrigin], residual: TileTensor[kernel_dtype, residual_LT, MutAnyOrigin], u: TileTensor[kernel_dtype, u_LT, MutAnyOrigin], delta: TileTensor[kernel_dtype, delta_LT, MutAnyOrigin], A: TileTensor[kernel_dtype, A_LT, MutAnyOrigin], B: TileTensor[kernel_dtype, B_LT, MutAnyOrigin], C: TileTensor[kernel_dtype, C_LT, MutAnyOrigin], D: TileTensor[kernel_dtype, D_LT, MutAnyOrigin], z: TileTensor[kernel_dtype, z_LT, MutAnyOrigin], delta_bias: TileTensor[kernel_dtype, delta_bias_LT, MutAnyOrigin], gamma: TileTensor[kernel_dtype, gamma_LT, MutAnyOrigin], epsilon: Scalar[kernel_dtype], weight_offset: Scalar[kernel_dtype], ctx: Optional[DeviceContext] = None)

CPU kernel for SSD combined operation.

Each worker processes one (batch, dim) pair, runs the selective scan recurrence over the sequence, adds the residual, scales by gamma + weight_offset, and optionally gates the result by silu(z).

Parameters:

• ​kernel_dtype (DType): Element type of the input and output tensors.
• ​DSTATE (Int): Number of SSM state elements per (batch, dim) pair.
• ​output_LT (TensorLayout): Memory layout of the output tensor.
• ​x_LT (TensorLayout): Memory layout of the x checkpoint tensor.
• ​out_z_LT (TensorLayout): Memory layout of the out_z gated output tensor.
• ​residual_LT (TensorLayout): Memory layout of the residual tensor.
• ​u_LT (TensorLayout): Memory layout of the u input tensor.
• ​delta_LT (TensorLayout): Memory layout of the delta time-step tensor.
• ​A_LT (TensorLayout): Memory layout of the A recurrence matrix.
• ​B_LT (TensorLayout): Memory layout of the B input projection tensor.
• ​C_LT (TensorLayout): Memory layout of the C output projection tensor.
• ​D_LT (TensorLayout): Memory layout of the D skip connection tensor.
• ​z_LT (TensorLayout): Memory layout of the z gating tensor.
• ​delta_bias_LT (TensorLayout): Memory layout of the delta_bias tensor.
• ​gamma_LT (TensorLayout): Memory layout of the gamma normalization scale tensor.

Args:

• ​batch (Int): Number of sequences processed in parallel.
• ​dim (Int): Hidden dimension, equal to the number of channels per sequence position.
• ​seqlen (Int): Number of timesteps in each sequence.
• ​group_size (Int): Number of dims per group; dims in the same group share B and C inputs.
• ​delta_softplus (Int8): Nonzero applies softplus to delta before the scan recurrence.
• ​output (TileTensor[kernel_dtype, output_LT, MutAnyOrigin]): Normalized output tensor of shape (batch, dim, seqlen), written.
• ​x (TileTensor[kernel_dtype, x_LT, MutAnyOrigin]): Checkpoint tensor of shape (batch, dim, n_chunks, 2*DSTATE) storing per-chunk cumulative A and B values, written.
• ​out_z (TileTensor[kernel_dtype, out_z_LT, MutAnyOrigin]): Gated output tensor of shape (batch, dim, seqlen) holding normalized * silu(z), written when z is present.
• ​residual (TileTensor[kernel_dtype, residual_LT, MutAnyOrigin]): Residual input tensor of shape (batch, dim, seqlen), read; added to the scan output before normalization.
• ​u (TileTensor[kernel_dtype, u_LT, MutAnyOrigin]): Selective scan input tensor of shape (batch, dim, seqlen), read.
• ​delta (TileTensor[kernel_dtype, delta_LT, MutAnyOrigin]): Time-step tensor of shape (batch, dim, seqlen), read.
• ​A (TileTensor[kernel_dtype, A_LT, MutAnyOrigin]): SSM recurrence matrix of shape (dim, DSTATE), read.
• ​B (TileTensor[kernel_dtype, B_LT, MutAnyOrigin]): SSM input projection of shape (batch, n_groups, DSTATE, seqlen), read.
• ​C (TileTensor[kernel_dtype, C_LT, MutAnyOrigin]): SSM output projection of shape (batch, n_groups, DSTATE, seqlen), read.
• ​D (TileTensor[kernel_dtype, D_LT, MutAnyOrigin]): Skip connection vector of shape (dim,), read; added as D * u to the scan output when present.
• ​z (TileTensor[kernel_dtype, z_LT, MutAnyOrigin]): Gating tensor of shape (batch, dim, seqlen), read; gates the normalized output via silu(z) when present.
• ​delta_bias (TileTensor[kernel_dtype, delta_bias_LT, MutAnyOrigin]): Bias vector of shape (dim,), read; added to delta before softplus when present.
• ​gamma (TileTensor[kernel_dtype, gamma_LT, MutAnyOrigin]): Per-dim normalization scale vector of shape (dim,), read; added to weight_offset to form the per-dim scale applied to the combined output.
• ​epsilon (Scalar[kernel_dtype]): RMSNorm smoothing constant added inside the rsqrt for numerical stability.
• ​weight_offset (Scalar[kernel_dtype]): Scalar added to gamma to form the per-dim normalization scale applied to the combined output.
• ​ctx (Optional[DeviceContext]): Device context used to drive the parallel worker loop (defaults to None).