For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python function
fold_state_pools
fold_state_pools()
max.pipelines.speculative.gated_delta_rollback.fold_state_pools(live_recurrent_pools, recurrent_row_ids, ring_pools, ring_row_ids, merged_offsets, num_accepted, num_draft_tokens, signal_buffers, verify_width)
Applies the verify’s accepted ring records to the live recurrent pool.
One launch per device covers every layer, and none at a verify width of zero.
A row longer than the verify window was committed by the verify itself,
so its records were never written and it folds nothing
(ring_accepted_lengths()).
-
Parameters:
-
- live_recurrent_pools (Sequence[BufferValue]) – Per-device recurrent pool, still pre-verify.
- recurrent_row_ids (Sequence[TensorValue]) – Per-device
[num_layers, batch_size]state rows. - ring_pools (Sequence[BufferValue]) – Per-device ring pool.
- ring_row_ids (Sequence[TensorValue]) – Per-device
[num_layers, batch_size]ring rows. - merged_offsets (TensorValue) –
[batch + 1]offsets of the verified sequence. - num_accepted (TensorValue) –
[batch]accepted draft tokens per request. - num_draft_tokens (TensorValue) – Scalar
Kon the batch’s device. - signal_buffers (Sequence[BufferValue]) – Used only to place the fold counts on each device.
- verify_width (TensorValue) – The verify’s width operand.
-
Return type:
-
None