For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.lib.generate

Fixed-batch generation, the lowest-level entry point to the pipeline API.

This runs a pipeline’s forward passes over a batch you choose, with nothing added on top of the pipeline API. The caller’s list of prompts is the batch: it is never split, re-packed, or chunked, and every GenerateIterator.step() runs exactly one pipeline.execute in the caller’s thread. That also makes it useful for testing, such as reproducing and localizing kernel and model accuracy bugs. Serving features (scheduling and in-flight batching, chunked prefill, prefix caching, LoRA, and structured output) are tested through max serve. Kernels are not batch-invariant, so for bit-exact comparisons between MAX runs, use batch size 1 or ignore_eos=True.

Classes

BatchPipeline	The pipeline surface GenerateIterator drives.
GenerateIterator	Steps a fixed batch of prompts through a pipeline, one execute at a time.
GenerateResult	The result of generate().

Functions

generate	Generates tokens for a fixed batch of prompts, run as one batch.