For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

Speculator

Speculator

class max.pipelines.lib.arch_lookup.Speculator(name, base, draft_arch, method, pipeline_model, batching=None, weight_adapters=<factory>, example_repo_ids=None, opt_out_cascade=False, supports_device_graph_capture=None, config=None, memory_planner=None, checkpoint_draft_width=None, supports_spec_decode_mixed_batches=None)

source

Bases: object

One speculative variant of a target architecture.

A speculator is a bounded delta on its base, not a separate architecture: derive() copies only the fields below, and everything else – tokenizer, encodings, tool and reasoning parsers, structured-output defaults are inherited. The config class and memory planner are inherited unless the drafter adds KV cache or weights of its own. Declared in the speculator’s own package, which already depends on the base.

A speculator is not registered as an architecture. It is indexed against its target, and selection applies derive() to the architecture the checkpoint’s own name already resolved to.

Parameters:

• name (str)
• base (SupportedArchitecture)
• draft_arch (str | None)
• method (SpeculativeMethod)
• pipeline_model (PipelineModelType)
• batching (type[Any] | None)
• weight_adapters (Mapping[WeightsFormat, WeightsAdapter])
• example_repo_ids (list[str] | None)
• opt_out_cascade (bool)
• supports_device_graph_capture (bool | None)
• config (type[ArchConfig] | None)
• memory_planner (type[MemoryPlanner] | None)
• checkpoint_draft_width (Callable[[SpeculativeConfig, Any, Any], int] | None)
• supports_spec_decode_mixed_batches (bool | None)

base

base: SupportedArchitecture

source

The target architecture this speculator applies to.

batching

batching: type[Any] | None = None

source

checkpoint_draft_width

checkpoint_draft_width: Callable[[SpeculativeConfig, Any, Any], int] | None = None

source

Returns the draft width the drafter was trained for; None inherits.

config

config: type[ArchConfig] | None = None

source

Replace the base’s config class; None inherits.

Set it when the drafter keeps a KV cache of its own: the config’s KV params are what memory planning and cache allocation size, so a drafter group missing from them is never budgeted.

derive()

derive()

source

Returns the fused architecture for this speculator.

Applies this speculator’s own fields on top of base. Every field not named here is inherited, which is what keeps a speculator from drifting away from its target.

Return type:

SupportedArchitecture

draft_arch

draft_arch: str | None

source

The draft repo’s huggingface_config.architectures[0].

None when the draft head ships inside the target checkpoint, as for MTP/NextN: the draft manifest role stays empty and the draft weights come out of the target’s own state dict.

example_repo_ids

example_repo_ids: list[str] | None = None

source

Replace the target’s example repos.

Set it when the fused graph is exercised by a different checkpoint than the target’s own.

memory_planner

memory_planner: type[MemoryPlanner] | None = None

source

Replace the base’s memory planner; None inherits.

Set it when the fused graph loads drafter weights the base planner does not count.

method

method: SpeculativeMethod

source

The mechanism this speculator implements.

Matched against the configured speculative_method, so naming a method selects the mechanism that runs rather than whichever speculator the draft checkpoint happens to fit.

name

name: str

source

Identifies the fused architecture in logs and generated docs.

opt_out_cascade

opt_out_cascade: bool = False

source

Clear the base’s cascade_pipeline_factory.

The fused spec-decode graph has no cascade path, so inheriting the base’s factory would advertise one that does not exist.

pipeline_model

pipeline_model: PipelineModelType

source

The fused pipeline model, which runs target and draft in one graph.

supports_device_graph_capture

supports_device_graph_capture: bool | None = None

source

Override whether the fused graph can be captured; None inherits.

Set it False only for a fused graph that genuinely cannot be captured. A multimodal speculator whose vision encoder runs eagerly during prefill is the case that exists: it produces variable-shape image embeddings from outside the captured region, so the target’s own support does not carry over.

supports_spec_decode_mixed_batches

supports_spec_decode_mixed_batches: bool | None = None

source

Override whether the fused graph is per-row correct on mixed prefill+decode verify batches; None inherits.

weight_adapters

weight_adapters: Mapping[WeightsFormat, WeightsAdapter]

source