# MAX documentation

> High-performance AI serving and modeling for any hardware

Version: Nightly

For section-specific indexes, see:
- [MAX guides (index)](/llms-max-guides.txt)
- [MAX Python API (index)](/llms-python.txt)
- [MAX accelerator API (index)](/llms-accelerator-api.txt)
- [MAX C API (index)](/llms-c-api.txt)
- [MAX releases (index)](/releases-llms.txt)

Mojo language documentation is hosted separately at [mojolang.org](https://mojolang.org/llms.txt).


## Table of Contents

- [Quickstart](https://max.modular.com/get-started/): A quickstart guide to run a GenAI model locally with Modular.
- [Packages](https://max.modular.com/packages/): Learn how to install MAX tools, set up your environment, and choose between nightly and stable versions
- [MAX container](https://max.modular.com/container/): Learn more about the provided Docker container for MAX deployment.
- [Audio generation](https://max.modular.com/serve/audio-generation/): Generate music from a style caption and lyrics using the MAX v1/audio/speech endpoint
- [Benchmark MAX on NVIDIA or AMD GPUs](https://max.modular.com/serve/benchmark/): Learn how to use our benchmarking script to measure the performance of MAX
- [Embeddings](https://max.modular.com/serve/embeddings/): Learn how to use the MAX embeddings endpoint to create embeddings for input text
- [Function calling and tool use](https://max.modular.com/serve/function-calling/): Implement OpenAI-compatible function calling and tool use for agentic GenAI workflows
- [Image generation](https://max.modular.com/serve/image-generation/): Generate images from text prompts or transform existing images using the MAX v1/responses endpoint
- [Image and video to text](https://max.modular.com/serve/image-to-text/): Use the MAX chat completions endpoint with image or video input to generate descriptions and answer questions about visual content
- [Deploy MAX on GPU with self-hosted endpoints](https://max.modular.com/serve/local-to-cloud/): Learn how to deploy MAX pipelines to cloud
- [Using LoRA adapters](https://max.modular.com/serve/lora-adapters/): Use LoRA adapters with MAX to serve task-specific, fine-tuned variants of LLMs
- [Metrics](https://max.modular.com/serve/metrics/): Monitor MAX serving performance with Prometheus metrics and OpenTelemetry.
- [Parallelism](https://max.modular.com/serve/parallelism/): Distribute a model across multiple GPUs with tensor, data, or expert parallelism
- [Prefix caching](https://max.modular.com/serve/prefix-caching/): Enable prefix caching when serving a model with MAX
- [Reasoning](https://max.modular.com/serve/reasoning/): Serve reasoning models that separate chain-of-thought from the final answer.
- [Serve models with recipes](https://max.modular.com/serve/recipes/): Learn how to serve models with recipes that capture tuned max serve configurations for a specific model and hardware layout.
- [Speculative decoding](https://max.modular.com/serve/speculative-decoding/): Use speculative decoding to accelerate LLM inference
- [Structured output](https://max.modular.com/serve/structured-output/): Enable structured output with your GenAI deployments for predictable responses
- [Text to text](https://max.modular.com/serve/text-to-text/): Generate text using MAX with OpenAI-compatible chat and completion endpoints
- [Video generation](https://max.modular.com/serve/video-generation/): Generate videos from text prompts or animate existing images using the MAX v1/responses endpoint
- [Model development overview](https://max.modular.com/develop/): Learn the process to bring a trained model from Hugging Face to MAX.
- [max benchmark](https://max.modular.com/cli/benchmark/): Runs comprehensive benchmark tests on an active model server to measure
- [max encode](https://max.modular.com/cli/encode/): Converts input text into embeddings for semantic search, text similarity, and
- [max generate](https://max.modular.com/cli/generate/): Generates output from a given model and prompt without using an endpoint.
- [max](https://max.modular.com/cli/): The `max` command line tool runs and benchmarks MAX pipelines from one
- [max list](https://max.modular.com/cli/list/): Lists every pipeline architecture registered with MAX, along with example
- [max serve](https://max.modular.com/cli/serve/): Launches a model server with an OpenAI-compatible endpoint. Specify the
- [max warm-cache](https://max.modular.com/cli/warm-cache/): Preloads and compiles the model to optimize initialization time by:
- [max warm-interpreter-cache](https://max.modular.com/cli/warm-interpreter-cache/): MAX includes an interpreter that runs operations one at a time as your graph
- [max](https://max.modular.com/api/python/): The MAX Python API reference.
- [MAX accelerator library](https://max.modular.com/api/mojo/): Mojo APIs for hardware-accelerated programming: GPU primitives, compute kernels, tensor layouts, and graph extension points.
- [Using AI coding assistants](https://max.modular.com/coding-assistants/): Use AI coding assistants such as Cursor, Claude Code, Copilot, and Windsurf with Modular.
- [Basic operations](https://max.modular.com/develop/basic-ops/): Perform tensor operations for arithmetic, shape manipulation, reductions, and random generation
- [Broadcasting](https://max.modular.com/develop/broadcasting/): Learn broadcasting rules, automatic expansion in elementwise ops, and F.broadcast_to for explicit shapes
- [Build custom ops for GPUs](https://max.modular.com/develop/build-custom-ops/): Write custom GPU and CPU operations in Mojo and load them into a MAX graph.
- [Write hardware-agnostic custom ops for PyTorch](https://max.modular.com/develop/custom-kernels-pytorch/): Learn to write custom operators in Mojo for PyTorch
- [Optimize custom ops for GPUs with Mojo](https://max.modular.com/develop/custom-ops-matmul/): Learn to use Mojo's GPU programming abstractions to progressively optimize a matrix multiplication
- [Intro to custom ops](https://max.modular.com/develop/custom-ops/): Extend MAX Graph with custom Mojo kernels for optimized performance
- [Accuracy issues](https://max.modular.com/develop/debugging-accuracy/): Catch NaN and Inf values and uninitialized memory reads in your MAX model with accuracy debugging options.
- [Runtime errors](https://max.modular.com/develop/debugging-errors/): Capture Mojo stack traces and dump compiler IR to investigate unrecoverable failures in your MAX model.
- [GPU errors](https://max.modular.com/develop/debugging-gpu/): Force synchronous GPU execution and enable kernel-level bounds checking to pinpoint the op that caused a failure.
- [Execution tracing](https://max.modular.com/develop/debugging-tracing/): See which operations MAX runs, in what order, and on which device, and map each op back to its Python source.
- [Model debugging overview](https://max.modular.com/develop/debugging/): Review and configure available MAX debugging tools to diagnose issues during model development.
- [Data types (dtype)](https://max.modular.com/develop/dtypes/): Learn how to use dtypes to control tensor precision and interoperability
- [Eager execution](https://max.modular.com/develop/eager-execution/): Use the eager API to create tensors, apply operations, inspect intermediate values, and run a forward pass.
- [Graph overview](https://max.modular.com/develop/graph/): Learn why MAX uses graph compilation and how two programming patterns — explicit graph construction and eager-like execution — both produce graphs.
- [Indexing and slicing](https://max.modular.com/develop/indexing/): Use slice syntax, F.gather, F.where, and F.scatter to read and write tensor values
- [Layer comparison](https://max.modular.com/develop/layer-comparison/): Localize the first layer where a MAX model diverges from its PyTorch reference by comparing intermediate tensor outputs.
- [Write a custom layer](https://max.modular.com/develop/layers/): Write a custom layer for a model component that max.nn doesn't provide, and connect it to your model.
- [Logit comparison](https://max.modular.com/develop/logit-comparison/): Measure how closely a custom MAX model matches a PyTorch reference using mean absolute error, cosine distance, and KL divergence.
- [Serve a fine-tuned model on a supported architecture](https://max.modular.com/develop/max-pipeline-bring-your-own-model/): Serve a fine-tuned model checkpoint with MAX when the base architecture is already supported, with no architecture code required.
- [Bring up a model with MAX skills](https://max.modular.com/develop/model-bringup-skills/): Learn how to bring up new Hugging Face models in MAX using AI coding agents and MAX skills, including installation, prompting, and verification.
- [Model bring-up workflow](https://max.modular.com/develop/model-bringup-workflow/): Understand MAX's model architecture system and how to implement a custom architecture by mapping config fields, translating weight names, connectin...
- [Build a model graph with Module](https://max.modular.com/develop/modules/): Learn how to compose modules, write custom modules with explicit weights, load checkpoint data, and construct a model graph.
- [Model pipeline](https://max.modular.com/develop/pipelines/): Learn how to integrate your model into a pipeline for serving with MAX.
- [Quantization](https://max.modular.com/develop/quantize/): An introduction to the MAX Graph quantization API
- [Read a Hugging Face model config](https://max.modular.com/develop/read-model-config/): Translate Hugging Face config.json fields into MAX architecture decisions to identify what to reuse, subclass, or implement.
- [Serve your custom model](https://max.modular.com/develop/serve-custom-model-architectures/): Learn how to serve a model with a custom architecture with max serve and send inference requests to it.
- [Tensor realization](https://max.modular.com/develop/tensor-realization/): Understand how MAX realizes tensors so you can reason about eager performance and write code the graph compiler can optimize.
- [Tensor fundamentals](https://max.modular.com/develop/tensors/): This page provides an overview of tensors, why they matter, and how to create and work with them in MAX.
- [Weight adapters](https://max.modular.com/develop/weight-adapters/): Load a checkpoint into a MAX model by writing a weight adapter that reconciles names, dtypes, and key sets.
- [Environment variables](https://max.modular.com/environment-variables/): Reference for all configurable environment variables in MAX.
- [Intro to GPUs](https://max.modular.com/gpu/architecture/): An overview of GPU architecture and terminology.
- [GPU block and warp operations and synchronization](https://max.modular.com/gpu/block-and-warp/): A description of GPU block and warp operations and synchronization.
- [GPU debugging](https://max.modular.com/gpu/debugging/): Debugging Mojo programs on GPU.
- [GPU programming fundamentals](https://max.modular.com/gpu/fundamentals/): A description of the fundamental GPU APIs in Mojo.
- [Get started with GPU programming](https://max.modular.com/gpu/intro-tutorial/): Learn the basics of GPU programming with Mojo
- [GPU profiling with Nsight Systems](https://max.modular.com/gpu-system-profiling/): How to profile MAX models and endpoints with Nsight Systems.
- [Introduction to layouts](https://max.modular.com/layout/layouts/): Using layouts to describe multidimensional data.
- [Using LayoutTensor](https://max.modular.com/layout/tensors/): Using LayoutTensor to manipulate multidimensional data.
- [Supported models](https://max.modular.com/models/): See all the model architectures supported by MAX.
- [Create chat completion](https://max.modular.com/rest-api/create-chat-completion/): The POST /v1/chat/completions endpoint of the MAX REST API.
- [Create completion](https://max.modular.com/rest-api/create-completion/): The POST /v1/completions endpoint of the MAX REST API.
- [Create embeddings](https://max.modular.com/rest-api/create-embedding/): The POST /v1/embeddings endpoint of the MAX REST API.
- [Create response](https://max.modular.com/rest-api/create-response/): The POST /v1/responses endpoint of the MAX REST API.
- [Create speech](https://max.modular.com/rest-api/create-speech/): The POST /v1/audio/speech endpoint of the MAX REST API.
- [Health check](https://max.modular.com/rest-api/health/): The GET /health endpoint of the MAX REST API.
- [MAX REST API](https://max.modular.com/rest-api/): The API reference for the MAX inference endpoint.
- [List models](https://max.modular.com/rest-api/list-models/): The GET /v1/models endpoint of the MAX REST API.
- [Introduction to TileTensor](https://max.modular.com/tile-tensor/): Efficient abstractions for multidimensional data.
- [TileTensor layouts](https://max.modular.com/tile-tensor/layouts/): Using layouts to describe multidimensional data.
- [Using TileTensor](https://max.modular.com/tile-tensor/tensors/): Using TileTensor to manipulate multidimensional data.