For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Write hardware-agnostic custom ops for PyTorch
When working with PyTorch models, you might encounter performance bottlenecks in specific operations that could benefit from custom optimization. You might also want to experiment with novel GPU algorithms or implement cutting-edge research ideas that aren't yet available in standard frameworks. Rather than rewriting your entire model or switching frameworks, you can write high-performance kernels in Mojo and integrate them into your existing PyTorch workflows, enabling both optimization and experimentation in your familiar development environment.
This tutorial demonstrates how to enhance a PyTorch model by implementing a custom grayscale image conversion operation in Mojo. You'll discover how to keep your familiar PyTorch development experience while unlocking the performance benefits that Mojo provides for compute-intensive operations.
In this tutorial, you'll learn to convert an RGB image into a grayscale image by integrating a custom op in Mojo and running it in PyTorch.
Set up
Let's start by creating a Python project and installing the necessary tools.
- pixi
- uv
- If you don't have it, install
pixi:curl -fsSL https://pixi.sh/install.sh | shThen restart your terminal for the changes to take effect.
- Create a project:
pixi init pytorch_custom_ops \ -c https://conda.modular.com/max-nightly/ -c conda-forge \ && cd pytorch_custom_ops - Install
maxwith all dependencies (nightly) and other required packages:pixi add max-all torch pillow numpy - Start the virtual environment:
pixi shell
- If you don't have it, install
uv:curl -LsSf https://astral.sh/uv/install.sh | shThen restart your terminal to make
uvaccessible. - Create a project:
uv init pytorch_custom_ops && cd pytorch_custom_ops - Create and start a virtual environment:
uv venv && source .venv/bin/activate - Install
maxwith all dependencies (nightly) and other required packages:uv add "max[all]" torch pillow numpy \ --index https://whl.modular.com/nightly/simple/ \ --prerelease allow
This will install the max package along with PyTorch (torch), PIL for
image processing (pillow), and NumPy for array operations. When you install
the max package, you'll get access to the MAX Python APIs and the Mojo
compiler—everything needed to build high-performance custom operations.
Build the PyTorch interface
Let's start by creating the PyTorch side of our integration. We'll build a simple function that uses our custom operation, but first we need to establish the interface.
Create a new file called grayscale.py and add the following code:
from pathlib import Path
import torch
@torch.compile
def grayscale(pic: torch.Tensor):
output = pic.new_empty(pic.shape[:-1]) # Remove color channel dimension
# We'll call our custom operation here
return outputThe grayscale function transforms a color image (with red, green, and blue
channels) into a single-channel grayscale image. While PyTorch has built-in
operations for this, implementing it as a Mojo custom op demonstrates the
integration pattern you can apply to any performance-critical computation. Now
we need to bridge to our Mojo implementation.
Integrate the custom operation
The max.experimental.torch module provides
CustomOpLibrary,
which allows you to load and use compiled Mojo operations directly in PyTorch.
Update the grayscale.py file to include the following code:
from pathlib import Path
import torch
from max.experimental.torch import CustomOpLibrary
# Load the compiled Mojo package containing our custom operations
mojo_kernels = Path(__file__).parent / "operations"
ops = CustomOpLibrary(mojo_kernels)
@torch.compile
def grayscale(pic: torch.Tensor):
output = pic.new_empty(pic.shape[:-1]) # Remove color channel dimension
ops.grayscale(output, pic) # Call our Mojo custom op
return outputThe CustomOpLibrary loads operations from a precompiled Mojo package file
(.mojoc). Once loaded, you can call these operations just like any other
PyTorch function. The operations automatically handle data movement between
PyTorch and MAX, and integrate with PyTorch when needed.
The compilation of your Mojo code into a .mojoc file is handled when you run
your Python script. You don't need to manually invoke the Mojo compiler or
manage build steps.
Implement the Mojo kernel
Now for the core implementation—our high-performance Mojo kernel. We'll create a Mojo package that defines our grayscale conversion operation.
First, create an empty file called __init__.mojo inside the operations
folder. This file marks the folder as a Mojo package, which CustomOpLibrary
requires:
touch operations/__init__.mojoThen create a new file called grayscale.mojo inside the operations folder
and add the following code:
from std.sys import align_of
import extensibility
from max.algorithm import elementwise
from max.gpu.host import DeviceContext
from extensibility import Input, Output, Tensor, TileTensorable
from std.utils.coord import Coord, coord_to_index_list
@extensibility.register("grayscale")
struct Grayscale:
"""Convert RGB image tensor to grayscale using weighted formula."""
@staticmethod
def execute(
img_out: Some[Output & Tensor],
img_in: Some[Input & Tensor],
ctx: DeviceContext,
) raises:
"""Execute grayscale conversion on the input image tensor.
Args:
img_out: Output tensor for grayscale image (height x width).
img_in: Input tensor for RGB image (height x width x channels).
ctx: Device context for execution.
"""
comptime assert conforms_to(type_of(img_out), TileTensorable)
comptime assert conforms_to(type_of(img_in), TileTensorable)
comptime assert type_of(img_out).dtype == .uint8
comptime assert type_of(img_out).rank == 2
comptime assert type_of(img_in).dtype == .uint8
comptime assert type_of(img_in).rank == 3
var out_tile = img_out.to_tile_tensor[.uint8]()
var in_tile = img_in.to_tile_tensor[.uint8]().as_imm()
@inline(.always)
def color_to_grayscale[
simd_width: Int, element_alignment: Int = 1
](idx: Coord) {var out_tile, var in_tile}:
"""Convert one RGB pixel to grayscale using perceptual weighting.
Args:
idx: Index into the output tensor (row, col).
"""
comptime align = align_of[DType.uint8]()
def load(
row: Int, col: Int, channel: Int
) {imm in_tile} -> SIMD[.float32, simd_width]:
return in_tile.load[width=simd_width, alignment=align](
Coord(row, col, channel)
).cast[.float32]()
var idx_l = coord_to_index_list(idx)
var row = idx_l[0]
var col = idx_l[1]
# Load RGB values from input tensor
var r = load(row, col, 0)
var g = load(row, col, 1)
var b = load(row, col, 2)
# Apply standard grayscale conversion formula
# These weights are based on human visual perception
var gray = 0.21 * r + 0.71 * g + 0.07 * b
# Clamp to valid uint8 range, convert back, and store
out_tile.store[
width=simd_width,
alignment=min(
type_of(img_out).alignment, element_alignment * align
),
](idx, min(gray, 255).cast[.uint8]())
# Run the conversion in parallel over every output pixel
elementwise[simd_width=1](color_to_grayscale, img_out.shape(), ctx)First, the @extensibility.register("grayscale") decorator makes this operation
available to PyTorch under the name grayscale. This is the
ops.grayscale(output, pic) function called in the PyTorch model.
Each execute() argument's type is a generic parameter bounded by tensor
traits: Output & Tensor for a tensor the op writes and Input & Tensor for
one it reads. Some[...] is shorthand for declaring that parameter inline. The
comptime assert lines pin the dtype and rank each argument must have, and
to_tile_tensor() gives a TileTensor view of each one that the kernel can
index. as_imm() makes the input view read-only.
The color_to_grayscale closure takes a coordinate into the output tensor,
computes that pixel's grayscale value, and stores it in the output. The
elementwise() primitive calls it for every coordinate in the output shape and
parallelizes the work across available compute units.
Finally, elementwise() takes the device from ctx, so the same code runs on
both CPU and GPU.
This approach allows Mojo to handle the low-level optimization details: memory layout, vectorization, and parallelization, while you focus on the algorithm.
Run the example
Now that we have a PyTorch model and a Mojo kernel, we can test it with a real image.
Create a new file called main.py to the root of your project and add the
following code:
import io
import numpy as np
from PIL import Image
import torch
from grayscale import grayscale
# Specify a test image
image = Image.open("test_image.jpg")
# Convert to PyTorch tensor and move to GPU for processing
image_tensor = torch.from_dlpack(np.array(image)).cuda()
# Apply our custom grayscale operation
gray_image = grayscale(image_tensor)
# Convert back to PIL Image for display or further processing
result = Image.fromarray(gray_image.cpu().numpy())You can run the example with the following command:
python main.pyThe following image shows the result of the grayscale operation.

This example demonstrates the complete pipeline: downloading an image, converting it to a PyTorch tensor, processing it with our Mojo custom op, and converting back to a standard Python image format. The operation automatically leverages GPU acceleration when available, providing significant performance improvements over CPU-only implementations.
Next steps
Now that you wrote a Mojo kernel with your PyTorch model, check out these other tutorials:
- Build custom ops for GPUs: Write custom GPU and CPU operations in Mojo and load them into a MAX graph.
- Build a model graph with Module: Learn how to compose modules, write custom modules with explicit weights, load checkpoint data, and construct a model graph.