IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.

Skip to main content

For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

qmatmul_k

Provides CPU kernels for K-quant block-wise quantized matrix multiplication.

Functions

matmul_Q4_K: Computes a matrix multiplication with Q4_K block-quantized weights.
matmul_Q4_K_pack_b: Packs Q4_K quantized weights into the blocked layout consumed by the compute kernel.
matmul_Q6_K: Computes a matrix multiplication with Q6_K block-quantized weights.
matmul_Q6_K_pack_b: Packs Q6_K quantized weights into the blocked layout consumed by the compute kernel.

Functions