For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

quantization

Registers quantization and dequantization graph ops backed by the quantization and linalg kernels.

comptime values

logger

comptime logger = Logger(stdout, prefix=String(""), source_location=False)

Structs

• ​GGMLQ40Dequantize: Registers the ggml_q4_0_dequantize graph op with the graph compiler.
• ​GGMLQ4KDequantize: Registers the ggml_q4_k_dequantize graph op with the graph compiler.
• ​GGMLQ6KDequantize: Registers the ggml_q6_k_dequantize graph op with the graph compiler.
• ​QMatmulGPU_b4_g128: Registers the qmatmul_b4_g128 graph op with the graph compiler.
• ​QMatmulGPU_b4_g32: Registers the qmatmul_b4_g32 graph op with the graph compiler.
• ​QMatmulGPURepackGGUF: Registers the GGUF_gpu_repack_q4_0 graph op with the graph compiler.
• ​QMatmulGPURepackGPTQ_b4_g128: Registers the GPTQ_gpu_repack_b4_g128 graph op with the graph compiler.
• ​QMatmulGPURepackGPTQ_b4_g128_desc_act: Registers the GPTQ_gpu_repack_b4_g128_desc_act graph op with the graph compiler.
• ​QuantizeDynamicScaledFloat8: Registers the mo.quantize_dynamic_scaled_float8 graph op with the graph compiler.
• ​QuantizeStaticScaledFloat8: Registers the mo.quantize_static_scaled_float8 graph op with the graph compiler.
• ​QuantizeTensorDynamicScaledFloat8: Registers the mo.quantize_tensor_dynamic_scaled_float8 graph op with the graph compiler.
• ​ResizeBicubic: Registers the mo.resize.bicubic graph op with the graph compiler.
• ​ResizeLinear: Registers the mo.resize.linear graph op with the graph compiler.
• ​ResizeNearest: Registers the mo.resize.nearest graph op with the graph compiler.
• ​RMSNormFusedQuantizeDynamicScaledFP8: Registers the mo.composite.rms_norm_fused_quantize_dynamic_scaled_fp8 graph op with the graph compiler.
• ​Struct_block_scaled_preshuffle_b_5d: Run the AMD CDNA4 MXFP4 B 5D preshuffle as a custom op.
• ​Struct_block_scaled_preshuffle_scale_4d_per_expert: Per-step A-scale preshuffle for the AMD CDNA4 preb grouped matmul.
• ​Struct_dequant_mxfp4: Registers the mo.dequant.mxfp4 graph op with the graph compiler.
• ​Struct_dequant_mxfp6: Registers the mo.dequant.mxfp6 graph op with the graph compiler.
• ​Struct_grouped_quantize_dynamic_block_scaled: Registers the mo.grouped.quantize.dynamic.block.scaled graph op with the graph compiler.
• ​Struct_interleave_block_scales: Registers the mo.interleave.block.scales graph op with the graph compiler.
• ​Struct_quantize_dynamic_block_scaled: Registers the mo.quantize.dynamic.block.scaled graph op with the graph compiler.
• ​Struct_quantize_dynamic_block_scaled_mxfp4: Registers the mo.quantize.dynamic.block.scaled.mxfp4 graph op with the graph compiler.
• ​Struct_quantize_dynamic_block_scaled_mxfp6: Registers the mo.quantize.dynamic.block.scaled.mxfp6 graph op.
• ​Struct_unfused_qkv_matmul_ragged_paged_gguf_quantized: Registers the mo.unfused_qkv_matmul.ragged.paged.gguf_quantized graph op with the graph compiler.
• ​VroomQ40Matmul: Registers the vroom_q4_0_matmul graph op with the graph compiler.
• ​VroomQ40RepackWeights: Registers the vroom_q4_0_repack_weights graph op with the graph compiler.
• ​VroomQ4KMatmul: Registers the vroom_q4_k_matmul graph op with the graph compiler.
• ​VroomQ4KRepackWeights: Registers the vroom_q4_k_repack_weights graph op with the graph compiler.
• ​VroomQ6KMatmul: Registers the vroom_q6_k_matmul graph op with the graph compiler.
• ​VroomQ6KRepackWeights: Registers the vroom_q6_k_repack_weights graph op with the graph compiler.

Functions

• ​composite_rms_norm_fused_quantize_dynamic_scaled_fp8_shape: Computes the output shapes for the fused RMS norm and dynamic scaled FP8 quantization op.
• ​ggml_q4_0_dequantize_shape: Computes the output shape for the ggml_q4_0_dequantize graph op.
• ​ggml_q4_k_dequantize_shape: Computes the output shape for the ggml_q4_k_dequantize graph op.
• ​ggml_q6_k_dequantize_shape: Computes the output shape for the ggml_q6_k_dequantize graph op.
• ​GGUF_gpu_repack_q4_0_shape: Computes the output shape for the GGUF_gpu_repack_q4_0 graph op.
• ​GPTQ_gpu_repack_b4_g128_desc_act_shape: Computes the output shape for the GPTQ_gpu_repack_b4_g128_desc_act graph op.
• ​GPTQ_gpu_repack_b4_g128_shape: Computes the output shape for the GPTQ_gpu_repack_b4_g128 graph op.
• ​qmatmul_b4_g128_shape: Computes the output shape for the qmatmul_b4_g128 graph op.
• ​qmatmul_b4_g32_shape: Computes the output shape for the qmatmul_b4_g32 graph op.
• ​resize_bicubic_shape: Computes the output shape for the mo.resize.bicubic graph op.
• ​resize_linear_shape: Computes the output shape for the mo.resize.linear graph op.
• ​resize_nearest_shape: Computes the output shape for the mo.resize.nearest graph op.
• ​vroom_q4_0_matmul_shape: Computes the output shape for the vroom_q4_0_matmul graph op.
• ​vroom_q4_0_repack_weights_shape: Computes the output shape for the vroom_q4_0_repack_weights graph op.
• ​vroom_q4_k_matmul_shape: Computes the output shape for the vroom_q4_k_matmul graph op.
• ​vroom_q4_k_repack_weights_shape: Computes the output shape for the vroom_q4_k_repack_weights graph op.
• ​vroom_q6_k_matmul_shape: Computes the output shape for the vroom_q6_k_matmul graph op.
• ​vroom_q6_k_repack_weights_shape: Computes the output shape for the vroom_q6_k_repack_weights graph op.