For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo module
fp8_quantization
Provides FP8 quantization kernels supporting static, dynamic, and blockwise scaling.
comptime values
logger
comptime logger = Logger(stdout, prefix=String(""), source_location=False)
Functions
-
batched_quantize_dynamic_scaled_fp8: TileTensor primary implementation of batched dynamic scaled FP8 quantization. -
blockwise_scaled_fp8_with_epilogue: Our sm100 blockwise scaled fp8 matmul kernel still does not support fusion of elementwise operations. This is a temporary implementation that uses our sm100 blockwise scaled fp8 matmul kernel and dispatch a separate epilogue kernel to apply the elementwise operations. For non B200 GPUs, we use the naive blockwise scaled fp8 matmul which support normal epilogue natively. Callers must allocatec; when anelementwise_lambda_fnis supplied the matmul result is written intocand then read back by the lambda. -
convert_e4m3fn_to_e4m3fnuz: Convert E4M3FN weights to E4M3FNUZ format for AMD GPU compatibility. -
matmul_dynamic_scaled_fp8: TileTensor primary implementation of dynamic scaled FP8 matmul. -
max_reduction_scale_kernel: Per-row strided max-|x| reduction into a global FP8 scale. -
naive_blockwise_scaled_fp8_grouped_matmul: Dispatches the naive blockwise scaled FP8 grouped matmul kernel on the GPU. -
naive_blockwise_scaled_fp8_grouped_matmul_kernel: Computes one output element per thread for the naive blockwise scaled FP8 grouped matmul GPU kernel. -
naive_blockwise_scaled_fp8_matmul: Dispatches the naive blockwise scaled FP8 matmul kernel on the GPU. -
naive_blockwise_scaled_fp8_matmul_kernel: Computes one output element per thread for the naive blockwise scaled FP8 matmul GPU kernel. -
quantize_dynamic_scaled_fp8: TileTensor primary implementation of dynamic scaled FP8 quantization. -
quantize_static_scaled_fp8: TileTensor implementation of static scaled FP8 quantization. -
quantize_tensor_dynamic_scaled_fp8: TileTensor primary implementation of dynamic scaled FP8 quantization. -
zero_scale_global_kernel: Zeros the global FP8 scale factor in a single thread.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!