For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.speculative
Speculative decoding pipelines and configuration for MAX.
Submodules
max.pipelines.speculative.adaptive_width | Chooses the verify width that decodes the most tokens per second. |
|---|---|
max.pipelines.speculative.block_driver | Model-agnostic driver for block (DFlash / DSpark) speculative decoding. |
max.pipelines.speculative.depth_schedule | How many drafted tokens to verify, as a function of decode batch size. |
max.pipelines.speculative.driver | Model-agnostic driver for sequential (eagle / MTP) speculative decoding. |
max.pipelines.speculative.spec_target | The target-side contract a spec-decode driver verifies through. |
Configuration
RejectionSamplingStrategy | alias of Literal['greedy', 'residual', 'typical-acceptance', 'logit-comparison'] |
|---|---|
SpeculativeConfig | Configures speculative decoding for a pipeline. |
SpeculativeMethod | alias of Literal['eagle', 'mtp', 'dflash', 'dflash2'] |
VerifyWidthRange | One inclusive decode-batch-size range and the drafts to verify in it. |
Token merging
RaggedTokenMerger | Merges prompt and draft token sequences into a single ragged batch. |
|---|
ragged_token_merger | Builds a graph that merges prompt and draft tokens into a single ragged sequence. |
|---|