For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.speculative.gated_delta_rollback
Recurrent-state rollback for a Gated DeltaNet target’s spec decoding.
Rejecting a draft token has to un-advance every Gated DeltaNet recurrence in the target, and neither pool has a length pointer to move. The two leaves roll back by different mechanisms:
- Recurrent. The verify reads the live pool without writing it and
records each token’s update in a ring, a scratch leaf of the state cache.
Once acceptance is known,
fold_state_pools()applies the accepted records to the live pool in one launch per device. - Conv. The window is the last
kernel_size - 1raw inputs, so the verify leaves it unwritten andreplay_conv_pools()re-runs the conv kernel over the accepted rows of the copy the verify keeps of its input, one verify-window (K + 1) slot per row. Every op feeding it is causal or pointwise, so those rows are bit-identical to what a forward over the accepted prefix alone would have produced.
A window with no drafts has nothing to roll back, so at a verify width of zero
the verify’s forward lands both leaves on the live pools and every rollback op
launches nothing. A decode’s window is K + 1 tokens; a prompt mixed
into the batch runs longer than the verify window, and the verify commits
both leaves of such a row itself at its last real token, so it neither
replays nor folds anything and takes no slot of the copy.
Every pool is addressed as one buffer per leaf plus a
[num_layers, batch_size] tensor of the rows each layer occupies, and the
engine supplies both for the live leaves and the ring alike.
Replay inputs
ConvReplayInputs | What the conv replay reads from one layer's captured verify inputs. |
|---|
Window planning
accepted_lengths | Returns the [batch] rows of the verify window each request keeps. |
|---|---|
accepted_row_plan | Plans the replay's gather indices and ragged offsets. |
ring_accepted_lengths | Returns accepted_lengths(), zero for a row the verify committed. |
window_row_plan | Plans the replay over the verify's per-row window copy. |
Pool updates
fold_state_pools | Applies the verify's accepted ring records to the live recurrent pool. |
|---|---|
replay_conv_pools | Writes each conv window at the accepted length. |