For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Quickstart
MAX provides a high-performance engine for serving built-in and custom AI models. This quickstart shows you how to start a local endpoint and send it a text generation request using the OpenAI SDK.
Get startedβ
Let's set up your system to run MAX. Check that you meet the system
requirements. Then use pixi or uv to
create a project.
- pixi
- uv
-
If needed, install
pixi:curl -fsSL https://pixi.sh/install.sh | sh -
Create a project:
pixi init max-quickstart \ -c https://conda.modular.com/max-nightly/ -c conda-forge \ && cd max-quickstart -
Install
max-all:pixi add max-all
-
If needed, install
uv:curl -LsSf https://astral.sh/uv/install.sh | sh -
Create a project:
uv init max-quickstart && cd max-quickstart -
Install
max[all]:uv add "max[all]" \ --index https://whl.modular.com/nightly/simple/ \ --prerelease allow
Start the serverβ
Start a local server with max serve:
-
In your
max-quickstartdirectory, start the virtual environment:- pixi
- uv
pixi shellsource .venv/bin/activate -
Start the server:
max serve --model Qwen/Qwen3.8-27B -
Once the server starts, you'll see:
π Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)
Keep this terminal open while you send requests to the server.
Send a requestβ
Send a text generation request to the server.
-
Open a new terminal, go to your
max-quickstartdirectory, and start the virtual environment:- pixi
- uv
pixi shellsource .venv/bin/activate -
Save the following script as
generate-text.py:generate-text.pyfrom openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") prompt = "In one sentence, what is the KV cache in LLM inference?" completion = client.chat.completions.create( model="Qwen/Qwen3.8-27B", messages=[{"role": "user", "content": prompt}], ) print(completion.choices[0].message.content.strip()) -
Run the script:
python generate-text.py
You'll see a response similar to the following:
The KV cache stores the key and value vectors for previously processed tokens
so that autoregressive decoding can reuse them in attention without recomputing
them each step.Benchmark the modelβ
To measure the performance of your served model, use max benchmark. The following command runs a benchmark against the
chat completions endpoint using the Sonnet dataset:
max benchmark \
--model Qwen/Qwen3.8-27B \
--backend modular \
--endpoint /v1/chat/completions \
--dataset-name sonnet \
--num-prompts 500 \
--sonnet-input-len 550 \
--output-lengths 256 \
--sonnet-prefix-len 200 \
--max-concurrency 32When the benchmark finishes, MAX prints the results in your terminal.
To save the results as a JSON file, pass a path to --result-filename:
max benchmark \
... \
--result-filename "results/quickstart-benchmark.json"For a complete list of datasets and configuration options, see the
max benchmark reference.
Next stepsβ
MAX offers so much more than serving text generation models:
- Try out other inference tasks like image generation, video generation, and audio generation.
- Serve other models from the list of supported models or a custom model.
- Configure how MAX serves your model with the
max serveoptions. - Benchmark custom workloads and interpret the results with the benchmark tutorial.
- Build an app around an endpoint with the Agentic Cookbook.