IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

reducescatter_quant

Row reduce-scatter whose peer traffic is quantized (lossy).

Each GPU quantizes the rows it contributes to its peers' shards into the scratch that trails its Signal, in groups of 32 elements, then pulls its own shard's rows out of the peers' scratch and sums them, dequantized, in f32 with its own rows, which never leave the device and stay exact. Every element of a peer's contribution carries that group's quantization error, so the sum is not the bf16 reduce-scatter's; what it buys is fewer bytes over the links:

  • quant_bits == 8: E4M3 values with one E8M0 scale per group (MXFP8), 8.25 bits an element.
  • quant_bits == 4: symmetric INT4 in [-7, 7] with one bf16 scale per group, 4.5 bits an element.

comptime values​

RS_QUANT_GROUP​

comptime RS_QUANT_GROUP = 32

Functions​

Was this page helpful?