IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Build a model graph with Module

A Module holds your model's parameters and defines the computation that runs on them. The same forward() method runs eagerly on real tensors, traces to a Graph, or compiles into a callable the runtime executes, depending on the context you call it in.

You use modules to build a model definition, and to extend a MAX-supported model with custom layers. For a component the built-in layers don't provide, see Write a custom layer.

How a module runs​

A module declares the parameters it owns, then defines forward() over Tensor values. Operations come from max.functional, which you import as F.

A module runs in one of these ways:

  • Eager: calling the module executes forward() immediately on real tensors. This is how you check shapes and values on a small input.
  • Traced: trace() records the computation as a graph without compiling it. Each parameter becomes an external constant the graph refers to by name.
  • Compiled: compile() traces the module and compiles the graph into a callable, binding parameter values from a state dict.

The pipeline model compiles your module once at load time and calls the compiled result for every batch. It sets the default dtype while your _instantiate_module() builds the module and places it on a device, so layer constructors take shapes and options only.

Define and compose modules​

Compose a module from built-in layers when they cover what you need: Linear, Embedding, RMSNorm, LayerNorm, and others. Subclass Module directly and assign each child layer to an attribute. Here two Linear projections form a feed-forward block, and a ModuleList stacks two blocks:

from max.driver import CPU
from max import functional as F
from max.nn import Linear, Module, ModuleList
from max.tensor import Tensor, default_device


class FeedForward(Module[[Tensor], Tensor]):
    """Two linear projections with a SiLU activation between them."""

    def __init__(self, dim: int, hidden_dim: int) -> None:
        super().__init__()
        self.fc1 = Linear(dim, hidden_dim, bias=False)
        self.fc2 = Linear(hidden_dim, dim, bias=False)

    def forward(self, x: Tensor) -> Tensor:
        return self.fc2(F.silu(self.fc1(x)))


class TwoLayerStack(Module[[Tensor], Tensor]):
    """Two feed-forward blocks in a ModuleList."""

    def __init__(self, dim: int) -> None:
        super().__init__()
        self.layers = ModuleList([FeedForward(dim, dim) for _ in range(2)])

    def forward(self, x: Tensor) -> Tensor:
        for layer in self.layers:
            x = layer(x)
        return x


with default_device(CPU()):
    model = TwoLayerStack(dim=8)
    for name, _ in model.parameters:
        print(name)

Each parameter's name is its attribute path through the module tree, with ModuleList entries named by index:

layers.0.fc1.weight
layers.0.fc2.weight
layers.1.fc1.weight
layers.1.fc2.weight

Linear includes a bias by default. The example passes bias=False, so each projection has only a weight.

The @module_dataclass decorator generates a module's constructor from its field annotations. Write a custom layer shows that form.

Load weights into the module​

parameters yields each parameter under its dotted attribute path, such as layers.0.self_attn.q_proj.weight, and load_state_dict() reads a state dict keyed by those paths:

from max.driver import CPU
from max.nn import Linear
from max.tensor import Tensor, default_device


with default_device(CPU()):
    linear = Linear(4, 8, bias=False)

    # The module names its parameters for you.
    print([name for name, _ in linear.parameters])

    # Load a checkpoint's values by parameter name.
    linear.load_state_dict({"weight": Tensor.ones([8, 4])})

    out = linear(Tensor.ones([2, 4]))
    print(float(out.to_numpy()[0, 0]))

With every weight set to one, each output element sums the input's four ones:

['weight']
4.0

load_state_dict() raises KeyError when a parameter has no entry in the state dict. With its default strict=True, it also raises ValueError for an entry that matches no parameter, so a mismatched architecture fails before the model runs. Here the checkpoint has a bias that the layer doesn't:

from max.driver import CPU
from max.nn import Linear
from max.tensor import Tensor, default_device


with default_device(CPU()):
    linear = Linear(4, 8, bias=False)
    try:
        linear.load_state_dict(
            {"weight": Tensor.ones([8, 4]), "bias": Tensor.ones([8])}
        )
    except ValueError as error:
        print(error)

The error names the entries that matched no parameter:

load_state_dict did not use some weights: {'bias'}

A shape mismatch raises ValueError too, and so does a dtype mismatch. Pass auto_cast=True to cast between float32 and bfloat16 on load.

A checkpoint's names often differ from a module's parameter names. A weight adapter renames checkpoint keys to the module's parameter names.

Walk a module's parameters​

A module's parameter names are paths through its attributes. module.parameters yields every parameter in the module and its submodules as (name, tensor) pairs, where the name is the tensor's dotted path through the module, such as encoder.layer1.weight. module.local_parameters yields the tensors held directly on that module, keyed by attribute name. module.children yields the direct submodules, and module.descendants yields every module below it keyed by path.

A module is also a tree: each attribute is a child keyed by attribute name, modules, lists, tuples, dicts, and other containers are interior nodes, and every other value is a leaf, such as None or an integer. module.parameters yields the items of tree.paths(module, leaf=Tensor). The max.tree API provides methods for traversing and modifying that tree, such as:

  • tree.paths() returns a dict that maps each leaf's dotted path to the leaf.
  • tree.update() writes path-keyed values into the tree in place.
  • tree.map() builds a new tree with each leaf replaced by what a function returns.

Every walk takes leaf, a type, a tuple of types, or a predicate that says where the walk stops. The following example reads a module's paths with and without leaf=Tensor, then doubles every weight with tree.map():

from max import tree
from max.driver import CPU
from max.nn import Linear, Module
from max.tensor import Tensor, default_device


class Projection(Module[[Tensor], Tensor]):
    """One linear projection."""

    def __init__(self, dim: int) -> None:
        super().__init__()
        self.linear = Linear(dim, dim, bias=False)

    def forward(self, x: Tensor) -> Tensor:
        return self.linear(x)


with default_device(CPU()):
    model = Projection(4)
    model.load_state_dict({"linear.weight": Tensor.ones([4, 4])})

    # Without leaf, every attribute value is a leaf.
    print(list(tree.paths(model)))

    # With leaf=Tensor, the walk stops at tensors only.
    print(list(tree.paths(model, leaf=Tensor)))

    # The result keeps the structure: a Projection whose weights are doubled.
    doubled = tree.map(lambda t: t * 2.0, model, leaf=Tensor)
    weight = tree.paths(doubled, leaf=Tensor)["linear.weight"]
    print(type(doubled).__name__, float(weight.to_numpy()[0, 0]))

The walk without leaf also reaches the layer's non-tensor attributes, quant_config and bias, and every weight in the mapped copy is 2.0:

['linear.quant_config', 'linear.weight', 'linear.bias']
['linear.weight']
Projection 2.0

load_state_dict() reads the parameters with tree.paths(module, leaf=Tensor) and writes the loaded weights back with tree.update().

tree.map() returns a module of the same type with every tensor replaced, so one call builds a copy of a module for each device of a distributed layer. local_shards returns one tensor per device, and device picks one by index:

tree.map(lambda t: t.local_shards[device], module, leaf=Tensor, shared=True)

That call returns a copy whose tensors are each one device's shard. With shared=True, a tensor that two attributes reference maps once, so the copy keeps them as one object.

Trace a module to a graph​

Trace a module to inspect the MLIR that compile() compiles. The trace declares each parameter as an external constant that refers to the weight by name:

from max.driver import CPU
from max.dtype import DType
from max.nn import Linear
from max.tensor.tensor import Tensor, TensorType, default_device


with default_device(CPU()):
    linear = Linear(4, 8, bias=False)
    x_type = TensorType(DType.float32, [2, 4], device=linear.device)

    # Tracing builds the graph without compiling it.
    staged = linear.trace(x_type)
    for line in str(staged).splitlines():
        if "mo.constant.external" in line:
            print(line.strip())

    # Compiling produces a callable bound to the module's current weights.
    compiled = linear.compile(x_type)
    print(compiled(Tensor.ones([2, 4])).shape)

The trace declares the weight parameter as an external constant, and the compiled callable returns a tensor of the projected shape:

%1 = mo.constant.external [name = "weight", align = 1, device = #M.device_ref<"cpu", 0>] : !mo.tensor<[8, 4], f32>
[Dim(2), Dim(8)]

compile() takes the input types and, optionally, the weights. Without weights, it binds the module's current parameter values. A pipeline model passes the input types its batch_processor builds and the adapted checkpoint weights, and invokes the returned callable once per batch.

Next steps​

From here, go deeper on the pieces a module touches:

  • Write a custom layer: subclass a built-in layer, define one from scratch, or wrap a custom op.
  • Eager execution: run Tensor and F operations directly to check a forward pass.
  • Weight adapters: rename checkpoint keys to the module's parameter names before load_state_dict().
  • Model bring-up workflow: see how modules become a served architecture, from config mapping to the KV cache.

Was this page helpful?