For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python function
accepted_row_plan
accepted_row_plan()
max.pipelines.speculative.gated_delta_rollback.accepted_row_plan(merged_offsets, num_accepted, num_draft_tokens, total_rows, device, *, plan_rows=None)
Plans the replay’s gather indices and ragged offsets.
A row’s accepted length is its merged length minus the drafts it did not
accept, which is the prompt length during prefill (where there are no
drafts) and 1 + num_accepted during decode. One expression, no phase
branch. Rows are captured row for row; a row longer than the verify
window replays through window_row_plan().
The gather is deliberately not trimmed to the accepted total: that
length is only known on the device, and materializing it as a shape would
cost a device-to-host sync every step. Instead the index vector is
plan_rows long and the trailing entries repeat the last accepted row.
replay_offsets stops at the accepted total, so the conv kernel never
reaches those trailing entries.
-
Parameters:
-
- merged_offsets (TensorValue) –
[batch + 1]offsets of the verified sequence. - num_accepted (TensorValue) –
[batch]accepted draft tokens per request. - num_draft_tokens (TensorValue) – Scalar
Kondevice. - total_rows (Dim) – Row count of the captured tensors the replay gathers from.
- device (DeviceRef) – Device the plan is built on.
- plan_rows (Dim | None) – Length of the index vector, at least the accepted total.
Defaults to
total_rows.
- merged_offsets (TensorValue) –
-
Returns:
-
which row of the verify’s per-token tensors feeds each replay row, and the ragged offsets over them.
-
Return type:
-
(row_indices, replay_offsets)