For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

Buffer

`Buffer`

class max.driver.Buffer(dtype: max.dtype.DType, shape: collections.abc.Sequence[int], device: max.driver.Device | None = None, usage: max.driver.Usage = Usage.DEFAULT)

source

class max.driver.Buffer(dtype: max.dtype.DType, shape: collections.abc.Sequence[int], stream: max.driver.DeviceQueue, usage: max.driver.Usage = Usage.DEFAULT)

class max.driver.Buffer(shape: ndarray[writable=False], device: max.driver.Device)

Bases: object

A multi-dimensional buffer allocated on a device.

Allocates memory onto a given device with the provided shape and dtype. Buffers can be sliced to provide strided views of the underlying memory, but any buffers input into model execution must be contiguous.

Supports numpy-style slicing but does not currently support setting items across multiple indices.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.float32)

from max import driver
from max.dtype import DType

device = driver.Accelerator() if driver.accelerator_count() else driver.CPU()
device_buffer = driver.Buffer(shape=[2, 3], dtype=DType.float32, device=device)

Parameters:

dtype – The data type of buffer elements.
shape – The buffer shape as a sequence of positive, non-zero integers.
device – The device to allocate the buffer on. Defaults to the CPU.
stream – The queue to associate the buffer with.
usage – The allocation intent, see Usage. Defaults to Usage.DEFAULT.

`contiguous()`

contiguous()

source

Creates a contiguous copy of the parent buffer.

Parameters:: self (Buffer)
Return type:: Buffer

`copy()`

copy(device: max.driver.Device | None = None) → max.driver.Buffer

source

Creates a deep copy on an optionally given device.

If device is None (the default), a copy is created on the same device.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())
cpu_copy = cpu_buffer.copy()

from max import driver
from max.dtype import DType

device = driver.Accelerator() if driver.accelerator_count() else driver.CPU()
cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())
device_copy = cpu_buffer.copy(device=device)

Parameters:: device – The device to create the copy on. Defaults to None (same device).
Returns:: A new buffer that is a copy of this buffer.

`device`

property device

source

The device on which the buffer is resident.

`dtype`

property dtype

source

The data type of the buffer elements.

`element_size`

property element_size

source

The size of the element type in bytes.

`from_dlpack()`

static from_dlpack(array, *, copy=None)

source

Create a buffer from an object implementing the dlpack protocol.

This usually does not result in a copy, and the producer of the object retains ownership of the underlying memory.

Parameters:

array (Any) – An object that implements the dlpack protocol.
copy (bool | None) – Whether to create a copy of the data. Defaults to None, which copies only when the producer requires it.

Returns:

A new buffer that views or copies the data.

Return type:

Buffer

`from_numpy()`

static from_numpy(arr)

source

Creates a buffer from a provided numpy array on the host device.

The underlying data is not copied unless the array is noncontiguous. If it is, a contiguous copy will be returned.

Parameters:: arr (ndarray[tuple[Any, ...], dtype[Any]]) – The numpy array to wrap.
Returns:: A new buffer that views or copies the numpy array data.
Return type:: Buffer

`inplace_copy_from()`

inplace_copy_from(src)

source

Copy the contents of another buffer into this one.

These buffers may be on different devices. Requires that both buffers are contiguous and have same size.

Parameters:

self (Buffer)
src (Buffer)

Return type:

None

`is_contiguous`

property is_contiguous

source

Whether the buffer is contiguously allocated in memory.

Returns False if the buffer is a non-contiguous slice.

Currently, some situations that are contiguous are considered non-contiguous for the purposes of the engine, such as when a buffer has negative steps.

`is_host`

property is_host

source

Whether the buffer is host-resident.

Returns False for GPU buffers and True for CPU buffers.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())

print(cpu_buffer.is_host)

`item()`

item() → Any

source

Returns the scalar value at a given location.

Currently implemented only for zero-rank buffers. The return type is converted to a Python built-in type.

`memset()`

memset(value: int) → None

source

Fills the buffer’s bytes in place with a repeated byte value.

The fill is enqueued on the buffer’s device stream. Contiguous slices are supported and only their bytes are changed. The value is a byte pattern, not a typed element value; zero clears every byte.

Parameters:: value – The byte value, between 0 and 255 inclusive.
Raises:: ValueError – If the buffer is not contiguous, is a sub-byte slice that does not start and end on whole bytes, or the value is outside the byte range.

`mmap()`

static mmap(filename, dtype, shape, mode='copyonwrite', offset=0)

source

Creates a memory-mapped buffer from a binary file on disk.

The argument semantics follow ~numpy.memmap.

Parameters:

filename (PathLike[str] | str) – Path of the binary file to map.
dtype (DType) – Element type of the mapped data.
shape (ShapeType | int) – Shape of the mapped buffer, either a tuple or an int.
mode (np._MemMapModeKind) – Memory-map mode, following ~numpy.memmap semantics.
offset (int) – Byte offset into the file where the mapping starts.

Returns:

A new buffer backed by the memory-mapped file.

Return type:

Buffer

`num_elements`

property num_elements

source

Returns the number of elements in this buffer.

Rank-0 buffers have 1 element by convention.

`pinned`

property pinned

source

Whether the allocation landed in the device’s host memory space. Ask usage for what was requested.

`rank`

property rank

source

The rank of the buffer.

`scalar()`

scalar(value: Any, dtype: max.dtype.DType, device: max.driver.Device | None = None) → max.driver.Buffer

source

Creates a scalar value of a given dtype and value.

Parameters:

value – The scalar value.
dtype – The data type of the scalar.
device – The device to allocate the buffer on. Defaults to None, which allocates the buffer on the CPU.

Returns:

A new rank-0 buffer holding the scalar value.

`shape`

property shape

source

The shape of the buffer.

`stream`

property stream

source

The stream to which the buffer is bound.

`strides`

property strides

source

How far apart consecutive elements are, per dimension.

Counted in elements, not bytes.

Slicing does not move any data, so a slice keeps its parent’s strides. Narrow a 10-wide buffer to 6 columns and it still takes 10 elements to reach the next row:

parent = driver.Buffer(shape=[8, 10], dtype=DType.uint8)
rows = parent[:, :6]
rows.shape          # (8, 6)
rows.strides        # (10, 1) -- still 10 to the next row
rows.is_contiguous  # False

A buffer must be contiguous to be passed into model execution.

`to()`

to(device: max.driver.Device, *, after: max.driver.DeviceEvent | None = None) → max.driver.Buffer

source

Overloaded function.

Overload 1: to(device: Device, *, after: DeviceEvent | None = None) -> Buffer

Returns a buffer that’s guaranteed to be on the given device.

The buffer is only copied if the requested device is different from the device upon which the buffer is already resident. A buffer on another queue of the same device becomes a view on the default queue, ordered after after.

Parameters:

device (Device) – The device to move the buffer to.
after (DeviceEvent, optional) – Recorded on the buffer’s current queue. Required when the result is a view.

Raises:

ValueError – If the result is a view and after is not given.

Overload 2: to(devices: Sequence[Device], *, after: DeviceEvent | None = None) -> list[Buffer]

Returns a list of buffers that are guaranteed to be on the given devices.

The buffers are only copied if the requested devices are different from the device upon which the buffer is already resident.

`to_numpy()`

to_numpy()

source

Converts the buffer to a numpy array.

If the buffer is not on the host, a copy will be issued.

Parameters:: self (Buffer)
Return type:: ndarray[tuple[Any, …], dtype[Any]]

`usage`

property usage

source

Allocation intent. Slices and views report their parent’s usage.

`view()`

view(dtype, shape=None)

source

Return a new buffer with the given type and shape that shares the underlying memory.

If shape is not given, it is deduced by resizing the last axis to fit the new dtype. A rank-0 buffer has no axis to resize, so it keeps shape () and only accepts a dtype of the same size. A ValueError is raised when no shape can be deduced.

Parameters:

self (Buffer)
dtype (DType)
shape (Sequence[int] | None)

Return type:

Buffer

`zeros()`

zeros(shape: collections.abc.Sequence[int], dtype: max.dtype.DType, device: max.driver.Device | None = None, usage: max.driver.Usage = Usage.DEFAULT) → max.driver.Buffer

source

Allocates a buffer with all elements initialized to zero.

Parameters:

shape – The shape of the buffer.
dtype – The data type of the buffer.
device – The device to allocate the buffer on. Defaults to None (CPU).
usage – The allocation intent, see Usage. Defaults to Usage.DEFAULT.

Returns:

A new buffer filled with zeros.

Buffer​

contiguous()​

copy()​

device​

dtype​

element_size​

from_dlpack()​

from_numpy()​

inplace_copy_from()​

is_contiguous​

is_host​

item()​

memset()​

mmap()​

num_elements​

pinned​

rank​

scalar()​

shape​

stream​

strides​

to()​

to_numpy()​

usage​

view()​

zeros()​

`Buffer`

`contiguous()`

`copy()`

`device`

`dtype`

`element_size`

`from_dlpack()`

`from_numpy()`

`inplace_copy_from()`

`is_contiguous`

`is_host`

`item()`

`memset()`

`mmap()`

`num_elements`

`pinned`

`rank`

`scalar()`

`shape`

`stream`

`strides`

`to()`

`to_numpy()`

`usage`

`view()`

`zeros()`