For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

block_Q4_K

struct block_Q4_K

Represents a single Q4_K quantization block with grouped 4-bit values.

Fields

base_scale (Float16): The super-block scale factor.
base_min (Float16): The super-block minimum value.
q_scales_and_mins (Array[UInt8, Int(12)]): The 6-bit quantized per-group scales and mins, packed into bytes.
q_bits (Array[UInt8, Int(128)]): The 4-bit quantized values, two per byte.

comptime group_count = 8

The number of groups within a block.

comptime group_size = 32

The number of elements per quantization group.