IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

fold_state_pools

fold_state_pools()​

max.pipelines.speculative.gated_delta_rollback.fold_state_pools(live_recurrent_pools, recurrent_row_ids, ring_pools, ring_row_ids, merged_offsets, num_accepted, num_draft_tokens, signal_buffers, verify_width)

source

Applies the verify’s accepted ring records to the live recurrent pool.

One launch per device covers every layer, and none at a verify width of zero.

A row longer than the verify window was committed by the verify itself, so its records were never written and it folds nothing (ring_accepted_lengths()).

Parameters:

  • live_recurrent_pools (Sequence[BufferValue]) – Per-device recurrent pool, still pre-verify.
  • recurrent_row_ids (Sequence[TensorValue]) – Per-device [num_layers, batch_size] state rows.
  • ring_pools (Sequence[BufferValue]) – Per-device ring pool.
  • ring_row_ids (Sequence[TensorValue]) – Per-device [num_layers, batch_size] ring rows.
  • merged_offsets (TensorValue) – [batch + 1] offsets of the verified sequence.
  • num_accepted (TensorValue) – [batch] accepted draft tokens per request.
  • num_draft_tokens (TensorValue) – Scalar K on the batch’s device.
  • signal_buffers (Sequence[BufferValue]) – Used only to place the fold counts on each device.
  • verify_width (TensorValue) – The verify’s width operand.

Return type:

None