For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.context

Request contexts and supporting types for MAX pipelines.

A context carries one request’s state across generation steps: its prompt and generated tokens, sampling parameters, structured-output state, and the outputs produced so far.

Concrete context classes

TextContext	A base class for model context, specifically for Text model variants.
TextAndVisionContext	A base class for model context, specifically for Vision model variants.
PixelContext	A model-ready context for image/video generation requests.
AudioContext	A model-ready context for audio generation requests.

Generation status

GenerationStatus	Enum representing the status of a generation process in the MAX API.

Constants

FUTURE_TOKEN	Placeholder written to a generated-token slot whose value is still being computed on device under overlap scheduling.

Context protocols

BaseContext	Core interface for request lifecycle management across all of MAX, including serving, scheduling, and pipelines.

Type variables

BaseContextType	Type variable for generic programming with BaseContext implementations.
TextGenerationContextType	Type variable for text generation context types, constrained to TextContext.
VLMContextType	Type variable for VLM context types, constrained to TextAndVisionContext.
PixelGenerationContextType	Type variable for pixel generation context types, constrained to PixelContext.
AudioGenerationContextType	Type variable for audio generation context types, constrained to AudioContext.

Sampling

SamplingParams	Request specific sampling parameters that are only known at run time.
SamplingParamsInput	Input dataclass for creating SamplingParams instances.
SamplingParamsGenerationConfigDefaults	Default sampling parameter values extracted from a model's GenerationConfig.

Output types

TextGenerationOutput	Represents the output of a text generation operation.
GenerationOutput	Output container for image generation pipeline operations.
TextGenerationResponseFormat	Represents the response format specification for a text generation request.
LogProbabilities	Log probabilities for an individual output token.

Token management

TokenBuffer	A dynamically resizable container for managing token sequences.
ImageMetadata	Metadata about an image in the prompt.
Range	Represents a range with start and end indices.
TokenHashOverride	Content hash to use in place of a token when hashing KV-cache blocks.

TokenSlice	Type alias for arrays containing token IDs as 64-bit integers.

Video

SampledVideo	The frames kept from a video, and the source they came from.
VideoSource
VideoStreamInfo	Header facts about a video's first video stream.

decode_video_frames	Decodes a subsampled set of frames from a video.
open_video_container	Opens a video container for reading via PyAV, rejecting known-vulnerable codecs.
plan_frame_indices	Predicts which frames decode_video_frames() would keep.
probe_video	Reads a video's header without decoding any frame.
validate_video	Checks a video's header, decoding no frame.

Grammar and structured output

GrammarEnforcementState	Manages grammar enforcement state for constrained decoding.
GrammarEnforcementSnapshot	Captured grammar-enforcement state for rollback.
GrammarMatcher	Per-request grammar matcher stepped each decode step.
StructuredOutputRegionDelimiters	Token ID sequences that define structured output boundaries.

Speculative decoding

SpecDecodingState	Per-request state for speculative decoding.

EOS tracking

EOSTracker	Centralized EOS tracking: single-ID, sequence-ID, and stop-sequence checks.

Logits processors

LogitsProcessor	Callback that rewrites logits for a single sequence before sampling.
BatchLogitsProcessor	Callback that rewrites logits for a whole batch before sampling.

ProcessorInputs	Inputs passed to a logits processor callback.
BatchProcessorInputs	Arguments for a batch logits processor.

Exceptions

InputError	Exception raised for input validation errors that should be shown to users.
MediaBudgetExceeded	Raised when client-supplied media does not fit the request media budget.
MediaDecodeError	Raised when client-supplied media is malformed, unsupported, or unreadable.
PromptTooLongError	Raised when a prompt exceeds the model's maximum input length.

Validation functions

validate_aspect_ratio_args	Validates that required aspect ratio arguments are present for vision input.
validate_flux2_max_pixel_area	Rejects FLUX2 requests whose width * height exceeds the per-arch cap.
validate_image_grid_thw_args	Validates that image_grid_thw is present when vision encoding is needed.
validate_image_shape_5d	Validates that images have the expected 5-dimensional shape.
validate_initial_prompt_has_image	Validates that initial prompts contain an image for vision models.
validate_only_one_image	Validates that at most one image is provided in the context.
validate_requires_vision_context	Validates that the context is a TextAndVisionContext.
validate_vision_position_ids	Validates that vision_position_ids is present when vision encoding is needed.
validate_wan_max_pixel_area	Rejects WAN requests whose width * height exceeds the per-arch cap.