IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

matmul_Q4_K

def matmul_Q4_K(a_tt: TileTensor[.float32, Engine=a_tt.Engine, linear_idx_type=a_tt.linear_idx_type], b_tt: TileTensor[.uint8, Engine=b_tt.Engine, linear_idx_type=b_tt.linear_idx_type], c_tt: TileTensor[.float32, Engine=c_tt.Engine, linear_idx_type=c_tt.linear_idx_type], ctx: Optional[DeviceContext[CompilationTarget()]] = None)

Computes a matrix multiplication with Q4_K block-quantized weights.

Dispatches to an x86 or ARM NEON implementation at compile time; other targets fail to compile.

Args:

def matmul_Q4_K[EpilogueFnType: ElementwiseEpilogueFn](a_tt: TileTensor[.float32, Engine=a_tt.Engine, linear_idx_type=a_tt.linear_idx_type], b_tt: TileTensor[.uint8, Engine=b_tt.Engine, linear_idx_type=b_tt.linear_idx_type], c_tt: TileTensor[.float32, Engine=c_tt.Engine, linear_idx_type=c_tt.linear_idx_type], epilogue_fn: EpilogueFnType, ctx: Optional[DeviceContext[CompilationTarget()]] = None)

Same as the plain overload, also passing each output element to epilogue_fn.

The kernel uses c_tt as its accumulator, so it is written either way.

Parameters:

Args:

Was this page helpful?