IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.speculative.gated_delta_rollback

Recurrent-state rollback for a Gated DeltaNet target’s spec decoding.

Rejecting a draft token has to un-advance every Gated DeltaNet recurrence in the target, and neither pool has a length pointer to move. The two leaves roll back by different mechanisms:

  • Recurrent. The verify reads the live pool without writing it and records each token’s update in a ring, a scratch leaf of the state cache. Once acceptance is known, fold_state_pools() applies the accepted records to the live pool in one launch per device.
  • Conv. The window is the last kernel_size - 1 raw inputs, so the verify leaves it unwritten and replay_conv_pools() re-runs the conv kernel over the accepted rows of the copy the verify keeps of its input, one verify-window (K + 1) slot per row. Every op feeding it is causal or pointwise, so those rows are bit-identical to what a forward over the accepted prefix alone would have produced.

A window with no drafts has nothing to roll back, so at a verify width of zero the verify’s forward lands both leaves on the live pools and every rollback op launches nothing. A decode’s window is K + 1 tokens; a prompt mixed into the batch runs longer than the verify window, and the verify commits both leaves of such a row itself at its last real token, so it neither replays nor folds anything and takes no slot of the copy.

Every pool is addressed as one buffer per leaf plus a [num_layers, batch_size] tensor of the rows each layer occupies, and the engine supplies both for the live leaves and the ring alike.

Replay inputs​

ConvReplayInputsWhat the conv replay reads from one layer's captured verify inputs.

Window planning​

accepted_lengthsReturns the [batch] rows of the verify window each request keeps.
accepted_row_planPlans the replay's gather indices and ragged offsets.
ring_accepted_lengthsReturns accepted_lengths(), zero for a row the verify committed.
window_row_planPlans the replay over the verify's per-row window copy.

Pool updates​

fold_state_poolsApplies the verify's accepted ring records to the live recurrent pool.
replay_conv_poolsWrites each conv window at the accepted length.