For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.sampling

Token sampling for MAX pipelines: samplers, logits processors, and the acceptance runners used in speculative decoding.

Configuration

SamplingConfig	Configuration for the sampling stage of token generation.
ToolCallPolicy	Policy controlling constrained decoding for tool calls.

DEFAULT_STRUCTURED_OUTPUT_BACKEND	str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Processors

FrequencyData	Container for token frequency data in CSR format.
FusedSamplingProcessor	Applies sampling parameters to logits and stores the chosen tokens.
PenaltyInputs	Container for penalty inputs.
SamplerInputs	Container for sampler inputs.

Samplers

RejectionRunner	Interface for rejection sampling runners.
SyntheticRunner	Synthetic acceptance sampler for benchmarking.
TokenSampler	Samples tokens from the logits.

rejection_runner_registry	Given a rejection runner strategy, returns the type of RejectionRunner.
rejection_sampler	Builds a graph that implements speculative decoding rejection sampling.
rejection_sampler_with_residuals	Builds a rejection sampler with residual sampling for speculative decoding.
token_sampler	Builds a sampling graph that samples tokens from logits.

Logits processing

apply_logits_processors	Applies logits processors to a batch of logits.
build_greedy_acceptance_sampler_graph	Builds a graph that implements strict greedy acceptance for MTP.
build_stochastic_acceptance_sampler_graph	Builds a target-only stochastic rejection sampler for speculative decoding.
build_synthetic_acceptance_sampler_graph	Builds a graph that implements synthetic acceptance sampling.

Label scoring

LabelScorer	Gathers candidate-token log-probabilities from next-token logits.