IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Graph overview

Every model that runs in MAX starts as a graph, a representation of the model's computation that the compiler optimizes. You can build a graph explicitly with the max.graph API, or run eager code that stages operations into a graph, and both reach the same structure.

Why MAX compiles models to graphs​

To optimize a model for production deployment, MAX records the model's computation into a graph and compiles it. You execute a model by running the compiled graph. During graph compilation, the compiler can perform optimizations such as:

  • Fusing sequences of ops into single kernels to reduce memory traffic.
  • Planning memory allocation and reuse across the computation.
  • Dispatching hardware-tuned kernels that saturate your specific CPU or GPU.

This compilation step runs once, and every execution after it runs the optimized graph.

Other frameworks like PyTorch use an eager execution programming pattern. This works well for research and quick iteration, but for production inference it leaves less room to optimize across the full computation.

Anatomy of a graph​

In MAX, a Graph is the data structure that contains a model's computation. When operations (ops) run inside a graph, MAX records each operation into an intermediate representation (IR) that the compiler uses to analyze and optimize the computation. MAX doesn't execute any operations at this stage.

This diagram represents a graph that captures how data (including weights) flows through a linear-ReLU layer:

Figure 1. A linear-ReLU layer

In figure 1, matmul, add, and relu are different ops. Each op here accepts one or more inputs, records its computation as a node in the graph, and produces an output that the next op consumes.

These inputs and outputs—the values flowing through ops—are placeholders for tensors that will contain real data at execution time. You can query a placeholder for its shape, dtype, and device, but you can't read its data because no data exists yet. Once you compile the graph, you can run inference on real tensors.

Two ways to build a graph​

MAX supports two programming patterns for constructing a graph. Both patterns produce the same underlying graph structure but differ in how you define that structure. Start with modules: every ModuleV3 architecture uses them, and they run eagerly during development and compile for serving.

Modules and eager execution​

With max.tensor and max.functional, you write code that you can run directly. You define your model computation with operations on Tensor objects and execute that computation immediately without explicitly creating a graph.

This pattern is eager because every operation returns a result right away, even though MAX builds and runs a graph behind the call.

In fact, MAX stages eager operations into graphs (the same kind you'd build explicitly) and runs each staged graph so you get a result back. Every operation you run goes through this staging and realization work, which adds up in a production loop. For production deployment, group your ops into a Module and call compile() on the module to capture the whole computation as a single optimized graph that you can run quickly and repeatedly.

This example runs a ReLU operation with the eager API:

from max import functional as F
from max.tensor import Tensor

x = Tensor.ones([5])
y = F.relu(x)
print(y)  # Result is available immediately

Explicit graph construction​

With max.graph and max.nn.legacy, you build the graph directly. You open a Graph context, declare your inputs as TensorValue objects, apply ops to those symbolic values, and mark the outputs.

This pattern gives you full control over graph topology and is useful when you need to integrate custom ops or define multi-output graphs. It backs the architectures written with max.nn.legacy, and the graph that modules stage for you is the same structure. The tradeoff is that you can't inspect intermediate values during construction, and iteration is slower, because every TensorValue stays symbolic until you compile and run.

Here's an example using the Graph API to run a ReLU operation:

import numpy as np
from max.driver import CPU
from max.dtype import DType
from max.engine import InferenceSession
from max.graph import Graph, TensorType, ops

# Build the graph
cpu = CPU()
input_type = TensorType(DType.float32, shape=[5], device=cpu)

with Graph("relu_graph", input_types=[input_type]) as graph:
    x = graph.inputs[0]
    graph.output(ops.relu(x))

# Compile the graph
session = InferenceSession(devices=[cpu])
compiled = session.compile(graph)
model = session.init(compiled)

# Run the computation and inspect results
input_data = np.array([1.0, -2.0, 3.0, -4.0, 5.0], dtype=np.float32)
result = model.execute(input_data)
print(np.from_dlpack(result[0]))

Where the APIs differ​

The patterns use parallel but separate APIs:

Explicit graph constructionEager execution
Primary data objectmax.graph.TensorValuemax.tensor.Tensor
Built-in operationsmax.graph.opsmax.functional
Python operators+, -, *, /, @ on TensorValue+, -, *, /, @ on Tensor
Module abstractionmax.nn.legacy.Modulemax.nn.Module

Next steps​

  • Build a model graph with Module: Learn how to compose ops into reusable Module objects and assemble them into a full model graph.
  • Model bring-up workflow: See how a model graph fits inside a MAX model architecture package, from configuration and weight loading to serving.
  • Model pipeline: Connect your model to an inference pipeline that handles tokenization, batching, and request routing.

Was this page helpful?