For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Graph overview
Every model that runs in MAX starts as a graph, a representation of the model's
computation that the compiler optimizes. You can build a graph explicitly with
the max.graph API, or run eager code that stages operations into a graph, and
both reach the same structure.
Why MAX compiles models to graphs
To optimize a model for production deployment, MAX records the model's computation into a graph and compiles it. You execute a model by running the compiled graph. During graph compilation, the compiler can perform optimizations such as:
- Fusing sequences of ops into single kernels to reduce memory traffic.
- Planning memory allocation and reuse across the computation.
- Dispatching hardware-tuned kernels that saturate your specific CPU or GPU.
This compilation step runs once, and every execution after it runs the optimized graph.
Other frameworks like PyTorch use an eager execution programming pattern. This works well for research and quick iteration, but for production inference it leaves less room to optimize across the full computation.
Anatomy of a graph
In MAX, a Graph is the data
structure that contains a model's computation. When operations (ops) run inside
a graph, MAX records each operation into an intermediate representation (IR)
that the compiler uses to analyze and optimize the computation. MAX doesn't
execute any operations at this stage.
This diagram represents a graph that captures how data (including weights) flows through a linear-ReLU layer:
In figure 1, matmul, add, and relu are different ops. Each op here
accepts one or more inputs, records its computation as a node in the graph, and
produces an output that the next op consumes.
These inputs and outputs—the values flowing through ops—are placeholders for tensors that will contain real data at execution time. You can query a placeholder for its shape, dtype, and device, but you can't read its data because no data exists yet. Once you compile the graph, you can run inference on real tensors.
Two ways to build a graph
MAX supports two programming patterns for constructing a graph. Both patterns produce the same underlying graph structure but differ in how you define that structure. Start with modules: every ModuleV3 architecture uses them, and they run eagerly during development and compile for serving.
Modules and eager execution
With max.tensor and
max.functional, you write code that you can run
directly. You define your model computation with operations on
Tensor objects
and execute that computation immediately without explicitly creating a graph.
This pattern is eager because every operation returns a result right away, even though MAX builds and runs a graph behind the call.
In fact, MAX stages eager operations into graphs (the same kind you'd
build explicitly) and runs each staged graph so you get a result back. Every
operation you run goes through this staging and realization work, which adds up
in a production loop. For production deployment, group your ops into a
Module and call
compile()
on the module to capture the whole computation as a single optimized graph that
you can run quickly and repeatedly.
This example runs a ReLU operation with the eager API:
from max import functional as F
from max.tensor import Tensor
x = Tensor.ones([5])
y = F.relu(x)
print(y) # Result is available immediatelyExplicit graph construction
With max.graph and
max.nn.legacy, you build the graph directly. You
open a Graph context, declare your
inputs as TensorValue objects,
apply ops to those symbolic values, and mark the
outputs.
This pattern gives you full control over graph topology and is useful when you
need to integrate custom ops or define multi-output graphs. It backs the
architectures written with max.nn.legacy, and the graph that modules stage for
you is the same structure. The tradeoff is that you can't inspect intermediate
values during construction, and iteration is slower, because every TensorValue
stays symbolic until you compile and run.
Here's an example using the Graph API to run a ReLU operation:
import numpy as np
from max.driver import CPU
from max.dtype import DType
from max.engine import InferenceSession
from max.graph import Graph, TensorType, ops
# Build the graph
cpu = CPU()
input_type = TensorType(DType.float32, shape=[5], device=cpu)
with Graph("relu_graph", input_types=[input_type]) as graph:
x = graph.inputs[0]
graph.output(ops.relu(x))
# Compile the graph
session = InferenceSession(devices=[cpu])
compiled = session.compile(graph)
model = session.init(compiled)
# Run the computation and inspect results
input_data = np.array([1.0, -2.0, 3.0, -4.0, 5.0], dtype=np.float32)
result = model.execute(input_data)
print(np.from_dlpack(result[0]))Where the APIs differ
The patterns use parallel but separate APIs:
| Explicit graph construction | Eager execution | |
|---|---|---|
| Primary data object | max.graph.TensorValue | max.tensor.Tensor |
| Built-in operations | max.graph.ops | max.functional |
| Python operators | +, -, *, /, @ on TensorValue | +, -, *, /, @ on Tensor |
Module abstraction | max.nn.legacy.Module | max.nn.Module |
Next steps
- Build a model graph with Module: Learn how to
compose ops into reusable
Moduleobjects and assemble them into a full model graph. - Model bring-up workflow: See how a model graph fits inside a MAX model architecture package, from configuration and weight loading to serving.
- Model pipeline: Connect your model to an inference pipeline that handles tokenization, batching, and request routing.