For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

Buffer

`Buffer`

class max.driver.Buffer(self, dtype: max.dtype.DType, shape: collections.abc.Sequence[int], device: max.driver.Device | None = None, usage: max.driver.Usage = Usage.DEFAULT)

source

class max.driver.Buffer(self, dtype: max.dtype.DType, shape: collections.abc.Sequence[int], stream: max.driver.DeviceQueue, usage: max.driver.Usage = Usage.DEFAULT)

class max.driver.Buffer(self, shape: ndarray[writable=False], device: max.driver.Device)

Bases: object

Device-resident buffer representation.

Allocates memory onto a given device with the provided shape and dtype. Buffers can be sliced to provide strided views of the underlying memory, but any buffers input into model execution must be contiguous.

Supports numpy-style slicing but does not currently support setting items across multiple indices.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.float32)

# Create a buffer on GPU
gpu = driver.Accelerator()
gpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.float32, device=gpu)

Parameters:

dtype (DType) – Data type of buffer elements.
shape (Sequence[int]) – Tuple of positive, non-zero integers denoting the buffer shape.
device (Device, optional) – Device to allocate buffer onto. Defaults to the CPU.
stream (DeviceQueue, optional) – Queue to associate the buffer with.
usage (Usage, optional) – Allocation intent, see Usage. Defaults to Usage.DEFAULT.

`contiguous()`

contiguous()

source

Creates a contiguous copy of the parent buffer.

Parameters:: self (Buffer)
Return type:: Buffer

`copy()`

copy(self, stream: max.driver.DeviceQueue) → max.driver.Buffer

source

copy(self, device: max.driver.Device | None = None) → max.driver.Buffer

Overloaded function.

copy(self, stream: max.driver.DeviceQueue) -> max.driver.Buffer

Creates a deep copy on the device associated with the queue.

Args:
stream (DeviceQueue): The queue to associate the new buffer with.

Returns:
Buffer: A new buffer that is a copy of this buffer.
copy(self, device: max.driver.Device | None = None) -> max.driver.Buffer

Creates a deep copy on an optionally given device.
If device is None (default), a copy is created on the same device.
```
from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())
cpu_copy = cpu_buffer.copy()

# Copy to GPU
gpu = driver.Accelerator()
gpu_copy = cpu_buffer.copy(device=gpu)
```
Args:
device (Device, optional): The device to create the copy on.
Defaults to None (same device).

Returns:
Buffer: A new buffer that is a copy of this buffer.

`device`

property device

source

Device on which tensor is resident.

`dtype`

property dtype

source

DType of constituent elements in tensor.

`element_size`

property element_size

source

Return the size of the element type in bytes.

`from_dlpack()`

from_dlpack(*, copy=None)

source

Create a buffer from an object implementing the dlpack protocol.

This usually does not result in a copy, and the producer of the object retains ownership of the underlying memory.

Parameters:

array (Any)
copy (bool | None)

Return type:

Buffer

`from_numpy()`

from_numpy()

source

Creates a buffer from a provided numpy array on the host device.

The underlying data is not copied unless the array is noncontiguous. If it is, a contiguous copy will be returned.

Parameters:: arr (ndarray[tuple[Any, ...], dtype[Any]])
Return type:: Buffer

`inplace_copy_from()`

inplace_copy_from(src)

source

Copy the contents of another buffer into this one.

These buffers may be on different devices. Requires that both buffers are contiguous and have same size.

Parameters:

self (Buffer)
src (Buffer)

Return type:

None

`is_contiguous`

property is_contiguous

source

Whether or not buffer is contiguously allocated in memory.

Returns false if the buffer is a non-contiguous slice.

Currently, we consider certain situations that are contiguous as non-contiguous for the purposes of our engine, such as when a buffer has negative steps.

`is_host`

property is_host

source

Whether or not buffer is host-resident.

Returns false for GPU buffers, true for CPU buffers.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())

print(cpu_buffer.is_host)

`item()`

item(self) → Any

source

Returns the scalar value at a given location.

Currently implemented only for zero-rank buffers. The return type is converted to a Python built-in type.

`mmap()`

mmap(dtype, shape, mode='copyonwrite', offset=0)

source

Parameters:

filename (PathLike[str] | str)
dtype (DType)
shape (ShapeType | int)
mode (np._MemMapModeKind)
offset (int)

Return type:

Buffer

`num_elements`

property num_elements

source

Returns the number of elements in this buffer.

Rank-0 buffers have 1 element by convention.

`pinned`

property pinned

source

Whether the allocation landed in the device’s host memory space. Ask usage for what was requested.

`rank`

property rank

source

Buffer rank.

`scalar`

scalar = <nanobind.nb_func object>

source

`shape`

property shape

source

Shape of buffer.

`stream`

property stream

source

Stream to which tensor is bound.

`strides`

property strides

source

How far apart consecutive elements are, per dimension.

Counted in elements, not bytes.

Slicing does not move any data, so a slice keeps its parent’s strides. Narrow a 10-wide buffer to 6 columns and it still takes 10 elements to reach the next row:

parent = driver.Buffer(shape=[8, 10], dtype=DType.uint8)
rows = parent[:, :6]
rows.shape          # (8, 6)
rows.strides        # (10, 1) -- still 10 to the next row
rows.is_contiguous  # False

A buffer must be contiguous to be passed into model execution.

`to()`

to(self, device: max.driver.Device) → max.driver.Buffer

source

to(self, stream: max.driver.DeviceQueue) → max.driver.Buffer

to(self, devices: collections.abc.Sequence[max.driver.Device]) → list[max.driver.Buffer]

to(self, streams: collections.abc.Sequence[max.driver.DeviceQueue]) → list[max.driver.Buffer]

Overloaded function.

to(self, device: max.driver.Device) -> max.driver.Buffer

Returns a buffer that’s guaranteed to be on the given device.

The buffer is only copied if the requested device is different from the device upon which the buffer is already resident.
to(self, stream: max.driver.DeviceQueue) -> max.driver.Buffer

Returns a buffer that’s guaranteed to be on the given device and associated with the given queue.

The buffer is only copied if the requested device is different from the device upon which the buffer is already resident. If the destination queue is on the same device, then a new reference to the same buffer is returned.
to(self, devices: collections.abc.Sequence[max.driver.Device]) -> list[max.driver.Buffer]

Returns a list of buffers that are guaranteed to be on the given devices.

The buffers are only copied if the requested devices are different from the device upon which the buffer is already resident.
to(self, streams: collections.abc.Sequence[max.driver.DeviceQueue]) -> list[max.driver.Buffer]

Returns a list of buffers that are guaranteed to be on the given queues.

The buffers are only copied if the requested queues are different from the queue upon which the buffer is already resident.

`to_numpy()`

to_numpy()

source

Converts the buffer to a numpy array.

If the buffer is not on the host, a copy will be issued.

Parameters:: self (Buffer)
Return type:: ndarray[tuple[Any, …], dtype[Any]]

`usage`

property usage

source

Allocation intent. Slices and views report their parent’s usage.

`view()`

view(dtype, shape=None)

source

Return a new buffer with the given type and shape that shares the underlying memory.

If the shape is not given, it will be deduced if possible, or a ValueError is raised.

Parameters:

self (Buffer)
dtype (DType)
shape (Sequence[int] | None)

Return type:

Buffer

`zeros`

zeros = <nanobind.nb_func object>

source

Buffer​

contiguous()​

copy()​

device​

dtype​

element_size​

from_dlpack()​

from_numpy()​

inplace_copy_from()​

is_contiguous​

is_host​

item()​

mmap()​

num_elements​

pinned​

rank​

scalar​

shape​

stream​

strides​

to()​

to_numpy()​

usage​

view()​

zeros​

`Buffer`

`contiguous()`

`copy()`

`device`

`dtype`

`element_size`

`from_dlpack()`

`from_numpy()`

`inplace_copy_from()`

`is_contiguous`

`is_host`

`item()`

`mmap()`

`num_elements`

`pinned`

`rank`

`scalar`

`shape`

`stream`

`strides`

`to()`

`to_numpy()`

`usage`

`view()`

`zeros`