For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
fp8_gemv_kernel
def fp8_gemv_kernel[c_type: DType, c_layout: TensorLayout, a_layout: TensorLayout, w_layout: TensorLayout, c_engine: TensorEngine, a_engine: TensorEngine, w_engine: TensorEngine, EpilogueFnType: ElementwiseEpilogueFn, has_epilogue_fn: Bool](c: TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine], a: TileTensor[.bfloat16, a_layout, ImmutAnyOrigin, Engine=a_engine], weight: TileTensor[.float8_e4m3fn, w_layout, ImmutAnyOrigin, Engine=w_engine], n_arg: Int32, k_arg: Int32, epilogue_fn: EpilogueFnType)
One warp per output column; 32 lanes stride down K widening FP8 -> f32.
c is [1, N], a the bf16 activation [1, K], weight the FP8-E4M3
weight [N, K]. Accumulation is fp32. Produces the RAW x @ W_fp8^T; the
per-tensor scalar weight_scale is folded post-matmul by the graph lowering.
With has_epilogue_fn, epilogue_fn stores each output in place of the
plain store to c.