IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

compress_windows

DeepSeek-V4's compressor over the windows a ragged chunk touches.

A compressed stream pools every ratio consecutive positions into one entry. A chunk of s tokens after P cached ones touches ceil(s / ratio) windows, the first starting at P - P % ratio; the batch's windows are laid end to end, request b owning output rows window_offsets[b] .. window_offsets[b + 1].

Window w pools coff * ratio positions: its own ratio and, when overlapping (coff == 2), its predecessor's. A position before P is read from the state leaf, which keeps every position's raw wkv (K) and wgate (V) rows; one at or after it from the chunk's projections. A window still open after the chunk is written as zeros without being pooled: its zone slot is unread until the window closes and a later chunk rewrites it. With m running over the pooled positions, output channel c is

entry[c] = sum_m softmax_m(wgate[m, c'] + ape[m % ratio, c']) * wkv[m, c']

where c' = c + (m // ratio) * head_dim when overlapping (the predecessor's low half, then the window's own high half) and c' = c otherwise. The entry is then finished as the reference finishes it: rounded to dtype, RMS-normed, its last rope_dim channels rotated at the window's first position, and snapped to the grid the model was trained against. The attention's entries take FP8 e4m3 on the non-rotary channels in groups of 64; the indexer's are rotated by the Hadamard matrix and take FP4 e2m1 in groups of 32. A group's scale is its absolute max over the format's max, rounded up to a power of two (UE8M0).

The window count is a sum of per-request ceilings, which the graph cannot size an output by without reading it back. The output's row count is an upper bound derived from shapes instead: the kernel reads the true count from window_offsets and leaves the rows past it unwritten, and the ragged store that consumes the entries skips them.

One block per window, vec channels per thread. A rotary pair stays in one thread and a quantization group in one warp, and the indexer's entry fits one warp for the Hadamard butterflies, so only the norm reduces across the block.

Functions​

Was this page helpful?