For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

fp8_gemv_kernel

def fp8_gemv_kernel[c_type: DType, c_layout: TensorLayout, a_layout: TensorLayout, w_layout: TensorLayout, elementwise_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None], c_engine: TensorEngine, a_engine: TensorEngine, w_engine: TensorEngine](c: TileTensor[c_type, c_layout, MutAnyOrigin, Engine=c_engine], a: TileTensor[.bfloat16, a_layout, ImmutAnyOrigin, Engine=a_engine], weight: TileTensor[.float8_e4m3fn, w_layout, ImmutAnyOrigin, Engine=w_engine], n_arg: Int32, k_arg: Int32)

One warp per output column; 32 lanes stride down K widening FP8 -> f32.

c is [1, N], a the bf16 activation [1, K], weight the FP8-E4M3 weight [N, K]. Accumulation is fp32. Produces the RAW x @ W_fp8^T; the per-tensor scalar weight_scale is folded post-matmul by the graph lowering.