IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

causal_conv1d_varlen_fwd_cpu

def causal_conv1d_varlen_fwd_cpu[dtype: DType, XFn: def[width: Int, alignment: Int](Int, Int) -> SIMD[dtype, width] & RegisterPassable & ImplicitlyCopyable, SlotFn: def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width] & RegisterPassable & ImplicitlyCopyable, //, silu_activation: Bool, use_residual: Bool = False, channels_last: Bool = False, ROW_RULE: Bool = False](weight: TileTensor[dtype, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], bias: TileTensor[dtype, Engine=bias.Engine, address_space=bias.address_space, linear_idx_type=bias.linear_idx_type], query_start_loc: TileTensor[.int32, Engine=query_start_loc.Engine, address_space=query_start_loc.address_space, linear_idx_type=query_start_loc.linear_idx_type], has_initial_state: TileTensor[.bool, Engine=has_initial_state.Engine, address_space=has_initial_state.address_space, linear_idx_type=has_initial_state.linear_idx_type], update_state: TileTensor[.bool, Engine=update_state.Engine, address_space=update_state.address_space, linear_idx_type=update_state.linear_idx_type], conv_states: TileTensor[Engine=conv_states.Engine, address_space=conv_states.address_space, linear_idx_type=conv_states.linear_idx_type], output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], x_fn: XFn, slot_fn: SlotFn, num_draft_tokens: Int = Int(0))

Forward pass for causal conv1d with variable length sequences.

Performs causal 1D convolution on variable length sequences that are concatenated together. Uses cumulative sequence lengths to identify sequence boundaries.

This is the CPU reference implementation for causal_conv1d_varlen_fwd.

Parameters:

  • ​dtype (DType): Element type of x, weight, bias and output.
  • ​XFn (def[width: Int, alignment: Int](Int, Int) -> SIMD[dtype, width] & RegisterPassable & ImplicitlyCopyable): Reads x at its physical (row, column) index, claiming alignment elements of alignment.
  • ​SlotFn (def[width: Int, alignment: Int](Int) -> SIMD[.uint32, width] & RegisterPassable & ImplicitlyCopyable): Reads the conv-state slot of each sequence.
  • ​silu_activation (Bool): Whether to apply the SiLU activation to the output.
  • ​use_residual (Bool): If True, adds x[d, s] to the convolution sum at each output position, before the activation.
  • ​channels_last (Bool): If True, x and output are (total_seqlen, dim) rather than (dim, total_seqlen).
  • ​ROW_RULE (Bool): If True, a prefill row (longer than one real token plus num_draft_tokens) writes its state window at its last real token, whatever its update_state flag.

Args:

Was this page helpful?