For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

apply_qk_rms_norm_cpu

def apply_qk_rms_norm_cpu[in_dtype: DType, out_dtype: DType, //](q_out: TileTensor[out_dtype, Engine=q_out.Engine, address_space=q_out.address_space, linear_idx_type=q_out.linear_idx_type], k_out: TileTensor[out_dtype, Engine=k_out.Engine, address_space=k_out.address_space, linear_idx_type=k_out.linear_idx_type], gamma_q: TileTensor[.float32, Engine=gamma_q.Engine, address_space=gamma_q.address_space, linear_idx_type=gamma_q.linear_idx_type], gamma_k: TileTensor[.float32, Engine=gamma_k.Engine, address_space=gamma_k.address_space, linear_idx_type=gamma_k.linear_idx_type], qk_var: TileTensor[.float32, Engine=qk_var.Engine, address_space=qk_var.address_space, linear_idx_type=qk_var.linear_idx_type], q: TileTensor[in_dtype, Engine=q.Engine, address_space=q.address_space, linear_idx_type=q.linear_idx_type], k: TileTensor[in_dtype, Engine=k.Engine, address_space=k.address_space, linear_idx_type=k.linear_idx_type], epsilon: Float32, rows: Int, q_cols: Int, k_cols: Int)

Naive CPU reference path (also used as a correctness oracle).