For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
indexer_qk_quantize
One-launch epilogues of a DSA lightning indexer with SSMax.
indexer_q_ssmax_rope_quantize replaces the query chain an SSMax indexer runs
after its query projection (SSMax scaling, interleaved RoPE over the leading
columns of each head, per-head FP8 quantization with an E8M0 scale), and
indexer_k_quantize_store replaces the whole key chain from the raw key
projection on (a biased LayerNorm, interleaved RoPE over the leading columns,
per-key FP8 quantization with an E8M0 scale, then the paged-cache stores of
the key and its scale).
Each step rounds where the graph-op chain it replaces rounds, so the outputs
are bit-identical to that chain: SSMax's factor is rounded to BF16 after the
log, after the scale and after the bias, the scaled query and the rotated
columns are rounded to BF16, the normalized key is rounded to BF16 before the
rotation, and the quantization derives its scale from the BF16 values exactly
as quantize_dynamic_scaled_fp8 does. Each row is computed on its own, so a
row's result does not depend on how many rows the launch holds.
The key's LayerNorm statistics are the bar's sharpest edge.
mo.reduce.layer_norm dispatches to the extensibility kernel
builtin_kernels.reductions.LayerNorm, which runs
builtin_kernels.rowwise_normalization.layer_norm; there
rowwise.pick_simd_width[Welford[float32, 1], target, 96, ...] picks the width.
The module's keep_dtype=False casts the key to float32 before the op, so that
pick is 4 -- float32's natural width, which the 96-byte budget leaves alone --
and 128 channels sit under the warp tier's ceiling, so every row takes one warp.
The reduction is Welford's online state: one four-wide fold per lane, a
sequential combine of the four lanes, then the warp's distance-halving
butterfly. _welford_fold/_welford_join below restate those two formulas
expression for expression, and the launch keeps the same lane-to-column map.
The tree's association order is what the byte comparison does not pin: the butterfly's stride order and the four-lane sequential combine alike. These keys are BF16 values upcast to float32 and every warp-level join combines equal counts, so its ratio is exactly one half and the statistics come out the same bit for bit whatever order the combines take -- a swapped stride order passes the tests unchanged. The order below mirrors the reference's by construction; the byte comparison pins the rounding points, not the association.
The key chain is also specified by the Moka component in
Kernels/test/gpu/Moka/kernels/indexer/indexer_k_tail_plan.mojo, whose plan and
sync certificate pin the launch the shipped graph op stages.
comptime values
INDEX_HEAD_DIM
comptime INDEX_HEAD_DIM = 128
Channels of one index head, which is also the quantization group.
Functions
-
indexer_k_quantize_store: Normalizes, rotates, FP8-quantizes index keys and stores them and their scales in the cache, in one launch. -
indexer_q_ssmax_rope_quantize: Scales, rotates and FP8-quantizes an indexer query in one launch.