IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

context_parallel

Token ownership for a KV cache sharded across a context-parallel group.

Under decode context parallelism (DCP) every rank of a group receives the same lookup table and the same global cache_lengths, but stores only the positions it owns. Position p belongs to rank (p // interleave) % size and lands in that rank's local slot (p // (interleave * size)) * interleave + p % interleave; local slots are addressed through the shared lookup table exactly like positions are without CP. A rank's local slots are ordered by global position, so "keys visible to a query at position q" is the prefix [0, local_len(q + 1)).

With size == 1 every function reduces to the identity, so kernels can call them unconditionally.

Structs​

Was this page helpful?