For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.speculative.spec_adapters
How spec decoding calls each target architecture’s module.
The modules MAX architectures build share no calling convention: each takes its
inputs in an order of its own and returns its outputs by position. A driver
can’t call a target without knowing which architecture built it, so each
target’s package holds a target adapter, its spec_adapters module, that
holds its call. _TARGET_ADAPTERS names the module of each target
architecture’s TargetAdapter, which takes what it leaves out from
Llama 3’s and holds a target’s differences under one speculative method in
by_method. A target whose ModuleV3 variant builds a module of another
kind names that variant’s adapter under the variant’s own name.
A module is imported only when one of its targets runs, so spec decoding imports no architecture it doesn’t run, and a build that doesn’t ship a private target’s package never imports it.
HISPARSE_INPUTS
max.pipelines.speculative.spec_adapters.HISPARSE_INPUTS = 'hisparse_inputs'
The key of a driver batch’s extra that holds the target’s HiSparse
graph inputs: the host and hot tables per device, then the host mirror, in
the order the signature declares them.
Only a target whose graph declares them reads the key; the draft never does, because its latent and indexer stay fully resident.
VerifyFn
max.pipelines.speculative.spec_adapters.VerifyFn
Verifies a driver batch on the target.
alias of Callable[[SequentialBatch | BlockBatch], Verified[Any, TensorValue]]
batch_distributed_buffers()
max.pipelines.speculative.spec_adapters.batch_distributed_buffers(target_arch, target_inputs)
Returns the per-device context lengths and the prepared buffers.
The buffers are the ones in ep_input_types() that the target’s batch
processor prepared.
None context lengths when it prepares none.
batch_replay_buffers()
max.pipelines.speculative.spec_adapters.batch_replay_buffers(target_arch, method, target_inputs)
Returns the replay inputs that the target prepared.
They are the batch’s values of replay_input_types(), and empty for the
full graph.
batch_trailing_buffers()
max.pipelines.speculative.spec_adapters.batch_trailing_buffers(target_arch, method, target_inputs)
Returns the trailing inputs that the target prepared.
They are the batch’s values of trailing_input_types(), for a target
whose batch processor prepares the pairing’s batches.
block_proposer()
max.pipelines.speculative.spec_adapters.block_proposer(target_arch, verify, draft, draft_config, speculative)
Returns the proposer of a draft that runs on its target’s own state.
None for every other draft, which its method’s proposer runs.
-
Parameters:
-
- target_arch (str)
- verify (Callable[[SequentialBatch | BlockBatch], Verified[Any, TensorValue]])
- draft (Module)
- draft_config (Any)
- speculative (SpeculativeConfig)
-
Return type:
-
BlockProposer[Any, Any, TensorValue, BufferValue, Value[Any]] | None
check()
max.pipelines.speculative.spec_adapters.check(target_arch, speculative, structured_output)
Raises if the target can’t verify under the speculative config.
-
Parameters:
-
- target_arch (str)
- speculative (SpeculativeConfig)
- structured_output (bool)
-
Return type:
-
None
ep_input_tree()
max.pipelines.speculative.spec_adapters.ep_input_tree(target_arch, target)
Returns the group that names a target’s expert-parallel inputs.
It is None when no group names them.
ep_input_types()
max.pipelines.speculative.spec_adapters.ep_input_types(target_arch, target)
Returns the expert-parallel inputs of a built target module.
They are the leaves of ep_input_tree() when a group names them.
-
Parameters:
-
Return type:
ep_tail()
max.pipelines.speculative.spec_adapters.ep_tail(target_arch, target)
Returns whether the target declares them on an undistributed signature.
export_contract()
max.pipelines.speculative.spec_adapters.export_contract(target_arch, speculative, draft_config, target_config, kv_params, input_types, structured_output)
Returns the record a fused-graph engine checks its inputs against.
Only a target whose engine keeps such a record has one.
An exporter writes it to the graph’s sidecar.
-
Parameters:
-
- target_arch (str)
- speculative (SpeculativeConfig)
- draft_config (Any)
- target_config (Any)
- kv_params (MultiKVCacheParams)
- input_types (Sequence[TensorType | BufferType])
- structured_output (bool)
-
Return type:
extra()
max.pipelines.speculative.spec_adapters.extra(target_arch, method, target_config, trailing)
Returns what the target’s call reads of the trailing graph inputs.
The driver hands it to the verify and the proposer as the batch’s
extra.
fold_index_glue()
max.pipelines.speculative.spec_adapters.fold_index_glue(target_arch)
Returns whether the target’s accept path folds its index glue.
A target opts into the folded single-launch ops; the default keeps the whole-tensor op chains.
has_adapter()
max.pipelines.speculative.spec_adapters.has_adapter(target_arch)
Returns whether spec decoding holds a call under exactly this name.
A ModuleV3 variant counts only when it names an adapter of its own.
hisparse_input_types()
max.pipelines.speculative.spec_adapters.hisparse_input_types(target_arch, method, target_config)
Returns the target’s HiSparse inputs, which follow the EP inputs.
Empty when the target keeps every latent page resident.
-
Parameters:
-
Return type:
prepares_every_batch()
max.pipelines.speculative.spec_adapters.prepares_every_batch(target_arch)
Returns whether the target prepares every batch of a pairing.
The inputs past the canonical tail count.
primary_kv()
max.pipelines.speculative.spec_adapters.primary_kv(target_arch, method)
Returns the name of the KV leaf a driver dispatches the target over.
The target’s other leaves ride in the driver batch’s passthrough_kv,
under their names in the KV tree. A target with one leaf has the name
"".
relaxed_thinking_acceptance()
max.pipelines.speculative.spec_adapters.relaxed_thinking_acceptance(target_arch)
Returns whether a reasoning model’s drafts may be accepted.
The relaxed in-thinking rule decides it.
replay_enabled()
max.pipelines.speculative.spec_adapters.replay_enabled(target_arch, method, pipeline_config, target_config)
Returns whether the pairing compiles a replay graph beside the full one.
replay_extra()
max.pipelines.speculative.spec_adapters.replay_extra(target_arch, method, target_config, replay)
Returns what the target’s call reads of the replay graph’s inputs.
replay_input_types()
max.pipelines.speculative.spec_adapters.replay_input_types(target_arch, method, target_config)
Returns the inputs only the replay graph takes, last of all.
-
Parameters:
-
Return type:
sequential_proposer()
max.pipelines.speculative.spec_adapters.sequential_proposer(target_arch, verify, draft, draft_config, speculative)
Returns the proposer of a one-token-per-step draft.
The draft runs on its target’s own module.
None for every other draft, which its method’s proposer runs.
-
Parameters:
-
- target_arch (str)
- verify (Callable[[SequentialBatch | BlockBatch], Verified[Any, TensorValue]])
- draft (Module)
- draft_config (Any)
- speculative (SpeculativeConfig)
-
Return type:
-
SequentialProposer[Any, TensorValue, BufferValue, Value[Any]] | None
signal_buffer_index()
max.pipelines.speculative.spec_adapters.signal_buffer_index(target_arch)
Returns where a fused graph takes its signal buffers.
The result is fixed across device counts, or none when the graph does not fix it.
speculates_on_last_stage()
max.pipelines.speculative.spec_adapters.speculates_on_last_stage(target_arch, method)
Returns whether method builds as the last pipeline stage.
The answer covers the target’s fused graph.
successor_replay()
max.pipelines.speculative.spec_adapters.successor_replay(target_arch, method)
Returns whether the draft head reads each position’s successor.
An engine then replays a cached prompt row’s boundary token, as Mach’s artifact boot decides from the export’s record.
trailing_buffers()
max.pipelines.speculative.spec_adapters.trailing_buffers(target_arch, method, target_config, contexts, devices)
Returns the trailing inputs that no KV child holds.
They are a batch’s values of trailing_input_types().
-
Parameters:
-
- target_arch (str) – The target checkpoint’s architecture name.
- method (str) – The speculative method.
- target_config (Any) – The config the target was built from.
- contexts (Sequence[TextContext]) – The batch, in the order its tokens are merged in.
- devices (Sequence[Device]) – The devices the target runs on.
-
Return type:
trailing_input_tree()
max.pipelines.speculative.spec_adapters.trailing_input_tree(target_arch, method, target_config)
Returns the groups that name trailing_input_types().
None for a target whose trailing inputs an exported graph leaves
unnamed.
trailing_input_types()
max.pipelines.speculative.spec_adapters.trailing_input_types(target_arch, method, target_config)
Returns the inputs the target’s call takes past the canonical tail.
-
Parameters:
-
Return type:
verify_commits_long_rows()
max.pipelines.speculative.spec_adapters.verify_commits_long_rows(target_arch, method)
Returns whether the target’s verify commits a long row.
A long row is longer than one token and its drafts, and the commit must
happen in every recurrent leaf under method. Mach’s gen_mef
writes the answer to the export’s sidecar.
verify_for()
max.pipelines.speculative.spec_adapters.verify_for(target_arch, target, target_config, hidden_states, draft_config)
Returns how a built target module verifies under a spec-decode driver.
-
Parameters:
-
- target_arch (str) – The target checkpoint’s architecture name.
- target (Module) – The module the target’s pipeline model built.
- target_config (Any) – The config it was built from.
- hidden_states (ReturnHiddenStates) – The hidden states it was built to return.
- draft_config (Any) – The config the draft was built from.
-
Return type:
-
Callable[[SequentialBatch | BlockBatch], Verified[Any, TensorValue]]
verify_takes_prefill_rows()
max.pipelines.speculative.spec_adapters.verify_takes_prefill_rows(target_arch, method, target_config)
Returns whether the verify commits a prefill row in a verify step.
An engine may then mix a prompt row into the step, as Mach’s artifact boot decides from the export’s record.