For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
cp_merge_topk
Merges a decode context-parallel group's local indexer top-k selections.
Under decode context parallelism (DCP) each rank of a group selects the top-k
of its own key slots (mla_indexer_ragged_float8_paged_cp), and the group
all-gathers the selections' scores, rank-major: rank r's k scores are
columns [r * k, (r + 1) * k) of a [T, dcp * k] matrix. The group's top-k
is the top-k of those dcp * k candidates, exactly up to ties, because a
token in the group's top-k is in its owner's local top-k. Every rank computes
the same global choice from the same matrix, then keeps the rows of its own
candidates that made it.
Per row, a two-pass radix select over the bf16 scores' order-preserving 16-bit
keys finds the k-th largest score t and how many candidates scoring exactly
t the top-k takes; those ties go to the lowest columns, so the choice is
deterministic and agreed on by every rank without communication.
Over an NVFP4 latent cache each selection carries a second row, its row in
the cache's scale pool, which the indexer writes top_k columns after the
value row (PagedKVScaleRemap). With halves = 2 a row of local_rows and
out_rows is 2 * top_k wide, and the merge moves each kept selection's
scale row with its value row, so both halves come out compacted the same way.
Functions
-
cp_merge_topk: Keeps this rank's candidates that are in its group's top-k.