For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
causal_conv1d_varlen_fwd_cpu
def causal_conv1d_varlen_fwd_cpu[dtype: DType, XFn: def[width: Int, alignment: Int](Int, Int) -> SIMD[dtype, width] & RegisterPassable & ImplicitlyCopyable, SlotFn: def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width] & RegisterPassable & ImplicitlyCopyable, //, silu_activation: Bool, use_residual: Bool = False, channels_last: Bool = False](weight: TileTensor[dtype, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], bias: TileTensor[dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type], query_start_loc: TileTensor[.int32, Engine=query_start_loc.Engine, address_space=query_start_loc.address_space, linear_idx_type=query_start_loc.linear_idx_type], has_initial_state: TileTensor[.bool, Engine=has_initial_state.Engine, address_space=has_initial_state.address_space, linear_idx_type=has_initial_state.linear_idx_type], update_state: TileTensor[.bool, Engine=update_state.Engine, address_space=update_state.address_space, linear_idx_type=update_state.linear_idx_type], conv_states: TileTensor[Engine=conv_states.Engine, address_space=conv_states.address_space, linear_idx_type=conv_states.linear_idx_type], output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], x_fn: XFn, slot_fn: SlotFn)
Forward pass for causal conv1d with variable length sequences.
Performs causal 1D convolution on variable length sequences that are concatenated together. Uses cumulative sequence lengths to identify sequence boundaries.
This is the CPU reference implementation for causal_conv1d_varlen_fwd.
Parameters:
- dtype (
DType): Element type ofx,weight,biasandoutput. - XFn (
def[width: Int, alignment: Int](Int, Int) -> SIMD[dtype, width]&RegisterPassable&ImplicitlyCopyable): Readsxat its physical(row, column)index, claimingalignmentelements of alignment. - SlotFn (
def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width]&RegisterPassable&ImplicitlyCopyable): Reads the conv-state slot of each sequence. - silu_activation (
Bool): Whether to apply the SiLU activation to the output. - use_residual (
Bool): If True, addsx[d, s]to the convolution sum at each output position, before the activation. - channels_last (
Bool): If True,xandoutputare(total_seqlen, dim)rather than(dim, total_seqlen).
Args:
- weight (
TileTensor[dtype, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type]): Weight tensor of shape(dim, width). - bias (
TileTensor[dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type]): Bias tensor of shape(dim,), or empty. - query_start_loc (
TileTensor[.int32, Engine=query_start_loc.Engine, address_space=query_start_loc.address_space, linear_idx_type=query_start_loc.linear_idx_type]): Cumulative sequence lengths of shape(batch + 1,). - has_initial_state (
TileTensor[.bool, Engine=has_initial_state.Engine, address_space=has_initial_state.address_space, linear_idx_type=has_initial_state.linear_idx_type]): Per-sequence flags of shape(batch,)selecting whether to continue the stored state, or empty. - update_state (
TileTensor[.bool, Engine=update_state.Engine, address_space=update_state.address_space, linear_idx_type=update_state.linear_idx_type]): Per-sequence flags of shape(batch,)selecting whether the sequence writes its new window toconv_states, or empty to write every sequence's. A clear flag leaves the slot untouched. - conv_states (
TileTensor[Engine=conv_states.Engine, address_space=conv_states.address_space, linear_idx_type=conv_states.linear_idx_type]): Convolution states of shape(max_slots, dim, width - 1), updated in place, or empty. - output (
TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): Output tensor of shape(dim, total_seqlen). - x_fn (
XFn): Reads the inputx, of shape(dim, total_seqlen). - slot_fn (
SlotFn): Reads each sequence's slot inconv_states.PAD_SLOT_IDskips the sequence.