For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
causal_conv1d_varlen_fwd_cpu
def causal_conv1d_varlen_fwd_cpu[dtype: DType, XFn: def[width: Int, alignment: Int](Int, Int) -> SIMD[dtype, width] & RegisterPassable & ImplicitlyCopyable, SlotFn: def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width] & RegisterPassable & ImplicitlyCopyable, //, silu_activation: Bool, use_residual: Bool = False, channels_last: Bool = False, ROW_RULE: Bool = False](weight: TileTensor[dtype, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], bias: TileTensor[dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type], query_start_loc: TileTensor[.int32, Engine=query_start_loc.Engine, address_space=query_start_loc.address_space, linear_idx_type=query_start_loc.linear_idx_type], has_initial_state: TileTensor[.bool, Engine=has_initial_state.Engine, address_space=has_initial_state.address_space, linear_idx_type=has_initial_state.linear_idx_type], update_state: TileTensor[.bool, Engine=update_state.Engine, address_space=update_state.address_space, linear_idx_type=update_state.linear_idx_type], conv_states: TileTensor[Engine=conv_states.Engine, address_space=conv_states.address_space, linear_idx_type=conv_states.linear_idx_type], output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], x_fn: XFn, slot_fn: SlotFn, num_draft_tokens: Int = Int(0))
Forward pass for causal conv1d with variable length sequences.
Performs causal 1D convolution on variable length sequences that are concatenated together. Uses cumulative sequence lengths to identify sequence boundaries.
This is the CPU reference implementation for causal_conv1d_varlen_fwd.
Parameters:
- dtype (
DType): Element type ofx,weight,biasandoutput. - XFn (
def[width: Int, alignment: Int](Int, Int) -> SIMD[dtype, width]&RegisterPassable&ImplicitlyCopyable): Readsxat its physical(row, column)index, claimingalignmentelements of alignment. - SlotFn (
def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width]&RegisterPassable&ImplicitlyCopyable): Reads the conv-state slot of each sequence. - silu_activation (
Bool): Whether to apply the SiLU activation to the output. - use_residual (
Bool): If True, addsx[d, s]to the convolution sum at each output position, before the activation. - channels_last (
Bool): If True,xandoutputare(total_seqlen, dim)rather than(dim, total_seqlen). - ROW_RULE (
Bool): If True, a prefill row (longer than one real token plusnum_draft_tokens) writes its state window at its last real token, whatever itsupdate_stateflag.
Args:
- weight (
TileTensor[dtype, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type]): Weight tensor of shape(dim, width). - bias (
TileTensor[dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type]): Bias tensor of shape(dim,), or empty. - query_start_loc (
TileTensor[.int32, Engine=query_start_loc.Engine, address_space=query_start_loc.address_space, linear_idx_type=query_start_loc.linear_idx_type]): Cumulative sequence lengths of shape(batch + 1,). - has_initial_state (
TileTensor[.bool, Engine=has_initial_state.Engine, address_space=has_initial_state.address_space, linear_idx_type=has_initial_state.linear_idx_type]): Per-sequence flags of shape(batch,)selecting whether to continue the stored state, or empty. - update_state (
TileTensor[.bool, Engine=update_state.Engine, address_space=update_state.address_space, linear_idx_type=update_state.linear_idx_type]): Per-sequence flags of shape(batch,)selecting whether the sequence writes its new window toconv_states, or empty to write every sequence's. A clear flag leaves the slot untouched; underROW_RULEa prefill row writes it anyway. - conv_states (
TileTensor[Engine=conv_states.Engine, address_space=conv_states.address_space, linear_idx_type=conv_states.linear_idx_type]): Convolution states of shape(max_slots, dim, width - 1), updated in place, or empty. - output (
TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type]): Output tensor of shape(dim, total_seqlen). - x_fn (
XFn): Reads the inputx, of shape(dim, total_seqlen). - slot_fn (
SlotFn): Reads each sequence's slot inconv_states.PAD_SLOT_IDskips the sequence. - num_draft_tokens (
Int): The step's draft-token count per row; the row rule commits a prefill row atrow_len - num_draft_tokens.