For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

LLM

`LLM`

class max.entrypoints.llm.LLM(pipeline_config)

source

Bases: object

A high-level interface for interacting with LLMs.

Use this class for offline batch inference against a model loaded from a PipelineConfig. Call generate() with one or more prompts to receive completions.

The following example loads LiquidAI/LFM2.5-350M and generates completions for a batch of prompts:

from max.entrypoints.llm import LLM
from max.pipelines import PipelineConfig

pipeline_config = PipelineConfig(model_path="LiquidAI/LFM2.5-350M")
llm = LLM(pipeline_config)

prompts = [
    "In the beginning, there was",
    "The meaning of life is",
]
responses = llm.generate(prompts, max_new_tokens=50)
for prompt, response in zip(prompts, responses, strict=True):
    print(prompt + response)

Parameters:: pipeline_config (PipelineConfig) – The PipelineConfig describing the model and runtime to load.

`generate()`

generate(prompts, max_new_tokens=100, use_tqdm=True)

source

Generates text completions for the given prompts.

This method is thread-safe and may be used on the same LLM instance from multiple threads concurrently with no external synchronization.

Parameters:

prompts (str | Sequence[str]) – The input string or list of strings to generate completions for.
max_new_tokens (int | None) – The maximum number of tokens to generate in the response. Defaults to 100.
use_tqdm (bool) – Whether to display a progress bar during generation. Defaults to True.

Returns:

A list of generated text completions corresponding to each input prompt.

Raises:

ValueError – If prompts is empty or contains invalid data.
RuntimeError – If the model fails to generate completions.

Return type:

Sequence[str]

LLM​

generate()​

`LLM`

`generate()`