For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

Speculator

Speculator

class max.pipelines.lib.arch_lookup.Speculator(name, base, draft_arch, method, pipeline_model, batching=None, weight_adapters=<factory>, example_repo_ids=None, opt_out_cascade=False, supports_device_graph_capture=None)

source

Bases: object

One speculative variant of a target architecture.

A speculator is a bounded delta on its base, not a separate architecture: derive() copies only the fields below, and everything else – tokenizer, config class, encodings, memory planner, tool and reasoning parsers, structured-output defaults are inherited. Declared in the speculator’s own package, which already depends on the base.

A speculator is not registered as an architecture. It is indexed against its target, and selection applies derive() to the architecture the checkpoint’s own name already resolved to.

Parameters:

• name (str)
• base (SupportedArchitecture)
• draft_arch (str | None)
• method (SpeculativeMethod)
• pipeline_model (PipelineModelType)
• batching (type[Any] | None)
• weight_adapters (Mapping[WeightsFormat, WeightsAdapter])
• example_repo_ids (list[str] | None)
• opt_out_cascade (bool)
• supports_device_graph_capture (bool | None)

base

base: SupportedArchitecture

source

The target architecture this speculator applies to.

batching

batching: type[Any] | None = None

source

derive()

derive()

source

Returns the fused architecture for this speculator.

Applies this speculator’s own fields on top of base. Every field not named here is inherited, which is what keeps a speculator from drifting away from its target.

Return type:

SupportedArchitecture

draft_arch

draft_arch: str | None

source

The draft repo’s huggingface_config.architectures[0].

None when the draft head ships inside the target checkpoint, as for MTP/NextN: the draft manifest role stays empty and the draft weights come out of the target’s own state dict.

example_repo_ids

example_repo_ids: list[str] | None = None

source

Replace the target’s example repos.

Set it when the fused graph is exercised by a different checkpoint than the target’s own.

method

method: SpeculativeMethod

source

The mechanism this speculator implements.

Matched against the configured speculative_method, so naming a method selects the mechanism that runs rather than whichever speculator the draft checkpoint happens to fit.

name

name: str

source

Identifies the fused architecture in logs and generated docs.

opt_out_cascade

opt_out_cascade: bool = False

source

Clear the base’s cascade_pipeline_factory.

The fused spec-decode graph has no cascade path, so inheriting the base’s factory would advertise one that does not exist.

pipeline_model

pipeline_model: PipelineModelType

source

The fused pipeline model, which runs target and draft in one graph.

supports_device_graph_capture

supports_device_graph_capture: bool | None = None

source

Override whether the fused graph can be captured; None inherits.

Set it False only for a fused graph that genuinely cannot be captured. A multimodal speculator whose vision encoder runs eagerly during prefill is the case that exists: it produces variable-shape image embeddings from outside the captured region, so the target’s own support does not carry over.

weight_adapters

weight_adapters: Mapping[WeightsFormat, WeightsAdapter]

source