IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

accepted_row_plan

accepted_row_plan()​

max.pipelines.speculative.gated_delta_rollback.accepted_row_plan(merged_offsets, num_accepted, num_draft_tokens, total_rows, device, *, plan_rows=None)

source

Plans the replay’s gather indices and ragged offsets.

A row’s accepted length is its merged length minus the drafts it did not accept, which is the prompt length during prefill (where there are no drafts) and 1 + num_accepted during decode. One expression, no phase branch. Rows are captured row for row; a row longer than the verify window replays through window_row_plan().

The gather is deliberately not trimmed to the accepted total: that length is only known on the device, and materializing it as a shape would cost a device-to-host sync every step. Instead the index vector is plan_rows long and the trailing entries repeat the last accepted row. replay_offsets stops at the accepted total, so the conv kernel never reaches those trailing entries.

Parameters:

  • merged_offsets (TensorValue) – [batch + 1] offsets of the verified sequence.
  • num_accepted (TensorValue) – [batch] accepted draft tokens per request.
  • num_draft_tokens (TensorValue) – Scalar K on device.
  • total_rows (Dim) – Row count of the captured tensors the replay gathers from.
  • device (DeviceRef) – Device the plan is built on.
  • plan_rows (Dim | None) – Length of the index vector, at least the accepted total. Defaults to total_rows.

Returns:

which row of the verify’s per-token tensors feeds each replay row, and the ragged offsets over them.

Return type:

(row_indices, replay_offsets)