For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.nn.legacy.graph_groups

Graph groups: the named inputs or outputs one model feature brings.

A graph’s input tree is a handful of groups, such as the ragged token batch, the allreduce signal buffers, or the vision embeddings, and its output tree likewise, such as the logits. Each group draws its fields from a fixed set, whatever model it appears in, so the serving engine learns to feed or read a group once, by its GraphGroup.kind, rather than once per model.

Graph groups

BatchContext	Per-device host scalars summarizing the batch's context lengths.
ConvStates	The short-convolution states a decoder writes back for the next step.
DecoderOutputs	A decoder graph's named outputs.
EPLBProfile	Per-layer expert counters an EPLB-profiling graph writes in place.
EPPayload	Expert-parallel communication state, built once at startup.
GraphGroup	The named graph inputs or outputs of one feature, in graph order.
HiddenStates	Hidden states a decoder returns for one mode.
Logits	The decoder's logits: the next token's, and any others requested.
LogitsName	The outputs a Logits group can hold.
MRoPEPositions	The 3D position ids of an M-RoPE model.
RaggedBatch	The step's ragged token stream: tokens, row offsets, and friends.
SparseIndices	The rows a sparse-attention indexer selected, for a drafter to reuse.
TPSignals	The allreduce signal buffers, one per device.
VisionMerge	Image embeddings per device and the token positions they replace.