For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.speculative.driver
Model-agnostic driver for sequential (eagle / MTP) speculative decoding.
Every sequential unified spec-decode module in the tree runs the same seven
phases – merge, verify, mask, accept, shift, propose, pack – and differs only
in how it calls its target, how it calls its draft, and what per-step cache
bookkeeping the draft needs. SequentialDriver owns the phases; a model
contributes a verify callable and a SequentialProposer.
The two adapters are deliberately not Module subclasses. The driver
registers the target and draft modules under the same attribute names the
hand-written modules used, so weight loading, state_dict keys and the
weights registry are unchanged.
The driver runs on graph values, but nothing outside it has to. A caller may
pass experimental Tensor inputs, one per
device, and gets Tensor outputs back; a proposer that subclasses
TensorAdapter, and a verify wrapped in tensor_compat_verify(), read and
return Tensor values too. Both convert at the boundary, so a ModuleV3
model never reaches for the graph API.
Draft state
CarryDimNames | How the driver names the carried hidden state's batch dim per step. |
|---|
DraftStepInput | Loop-carried state between draft steps. |
Proposed | One draft invocation's contribution. |
ReuseSpec | A per-step result the draft carries beside its hidden state. |
Draft cache
DecodeKVSwap | A single per-step edit to the draft's PagedCacheValues. |
|---|
DraftCache | Which cache the draft writes, and so what the driver advances. |
Experimental tensor adapters
TensorAdapter | Marks a proposer that runs on experimental Tensor values. |
|---|