For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
compress_windows
DeepSeek-V4's compressor over the windows a ragged chunk touches.
A compressed stream pools every ratio consecutive positions into one
entry. A chunk of s tokens after P cached ones touches
ceil(s / ratio) windows, the first starting at P - P % ratio; the
batch's windows are laid end to end, request b owning output rows
window_offsets[b] .. window_offsets[b + 1].
Window w pools coff * ratio positions: its own ratio and, when
overlapping (coff == 2), its predecessor's. A position before P is read
from the state leaf, which keeps every position's raw wkv (K) and
wgate (V) rows; one at or after it from the chunk's projections. A window
still open after the chunk is written as zeros without being pooled: its zone
slot is unread until the window closes and a later chunk rewrites it. With m
running over the pooled positions, output channel c is
entry[c] = sum_m softmax_m(wgate[m, c'] + ape[m % ratio, c']) * wkv[m, c']where c' = c + (m // ratio) * head_dim when overlapping (the predecessor's
low half, then the window's own high half) and c' = c otherwise. The entry
is then finished as the reference finishes it: rounded to dtype,
RMS-normed, its last rope_dim channels rotated at the window's first
position, and snapped to the grid the model was trained against. The
attention's entries take FP8 e4m3 on the non-rotary channels in groups of 64;
the indexer's are rotated by the Hadamard matrix and take FP4 e2m1 in groups
of 32. A group's scale is its absolute max over the format's max, rounded up
to a power of two (UE8M0).
The window count is a sum of per-request ceilings, which the graph cannot
size an output by without reading it back. The output's row count is an upper
bound derived from shapes instead: the kernel reads the true count from
window_offsets and leaves the rows past it unwritten, and the ragged store
that consumes the entries skips them.
One block per window, vec channels per thread. A rotary pair stays in one
thread and a quantization group in one warp, and the indexer's entry fits one
warp for the Hadamard butterflies, so only the norm reduces across the block.
Functions
-
compress_windows_ragged_paged: Pools every window of a ragged chunk into its compressed entry.