For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_matmul_int8_w8a8_apple
struct Struct_matmul_int8_w8a8_apple
Apple M5 int8 W8A8 matmul (no bias): out = dequant(quant(a) @ b^T).
A single FUSED graph op wrapping int8_matmul.mojo (online activation quant
- int8 widening-MMA GEMM + dequant, all internal). Inputs (matching
nn/kernels.py::_apple_int8_w8a8_matmulwithbias=None):abf16[M, K];bint8[N, K](transpose_b);b_scalefp32 per-channel[N](the Python squeezes the rowwise[N, 1]). Outputc_type[M, N]. The bias variant is the siblingmatmul.int8.w8a8.apple.biasop (custom-op input arity is fixed per registration, so the optional bias is a separate op name -- the same idiom askv_cache.fused_qkv_matmul.ragged.paged{,.bias}).
Implemented traitsโ
Methodsโ
executeโ
static def execute(c: T, a: T, b: T, b_scale: T, context: DeviceContext)