For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

AutoPipeline

AutoPipeline

class max.pipelines.AutoPipeline(model_path, *, max_batch_size=1, devices=None, device_graph_capture=False, **config_overrides)

source

Bases: object

Generates text from text with an AutoModel.

It encodes each prompt with an AutoTokenizer, generates, and decodes. Each output starts with its prompt exactly as passed. The prompt goes to the model as is, without a chat template.

pipe = AutoPipeline("HuggingFaceTB/SmolLM2-135M", max_batch_size=2)
texts = pipe(
    ["The capital of France is", "2 + 2 ="], max_new_tokens=16
)
print(texts)

Parameters:

• model_path (str) – A Hugging Face repo ID or a local model directory.
• max_batch_size (int) – The most prompts per data-parallel replica, as for AutoModel.
• devices (str | None) – The devices to load on, as for AutoModel.
• device_graph_capture (bool) – Whether decode steps replay captured graphs, as for AutoModel.
• **config_overrides (object) – Flat PipelineArgs field names. The tokenizer loads with the revision and trust_remote_code of the resolved config.

Raises:

• TypeError – As described for AutoModel.
• ValueError – As described for AutoModel.

close()

close()

source

Closes the model, as close() does.

Return type:

None

model

property model: AutoModel

source

The model that generates the tokens.

tokenizer

property tokenizer: AutoTokenizer

source

The tokenizer that encodes prompts and decodes outputs.