IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

Buffer

Buffer​

class max.driver.Buffer(dtype: max.dtype.DType, shape: collections.abc.Sequence[int], device: max.driver.Device | None = None, usage: max.driver.Usage = Usage.DEFAULT)

source

class max.driver.Buffer(dtype: max.dtype.DType, shape: collections.abc.Sequence[int], stream: max.driver.DeviceQueue, usage: max.driver.Usage = Usage.DEFAULT)

class max.driver.Buffer(shape: ndarray[writable=False], device: max.driver.Device)

Bases: object

Device-resident buffer representation.

Allocates memory onto a given device with the provided shape and dtype. Buffers can be sliced to provide strided views of the underlying memory, but any buffers input into model execution must be contiguous.

Supports numpy-style slicing but does not currently support setting items across multiple indices.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.float32)

# Create a buffer on GPU
gpu = driver.Accelerator()
gpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.float32, device=gpu)

Parameters:

  • dtype (DType) – Data type of buffer elements.
  • shape (Sequence[int]) – Tuple of positive, non-zero integers denoting the buffer shape.
  • device (Device, optional) – Device to allocate buffer onto. Defaults to the CPU.
  • stream (DeviceQueue, optional) – Queue to associate the buffer with.
  • usage (Usage, optional) – Allocation intent, see Usage. Defaults to Usage.DEFAULT.

contiguous()​

contiguous()

source

Creates a contiguous copy of the parent buffer.

Parameters:

self (Buffer)

Return type:

Buffer

copy()​

copy(device: max.driver.Device | None = None) → max.driver.Buffer

source

Creates a deep copy on an optionally given device.

If device is None (default), a copy is created on the same device.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())
cpu_copy = cpu_buffer.copy()

# Copy to GPU
gpu = driver.Accelerator()
gpu_copy = cpu_buffer.copy(device=gpu)

Parameters:

device (Device, optional) – The device to create the copy on. Defaults to None (same device).

Returns:

A new buffer that is a copy of this buffer.

Return type:

Buffer

device​

property device

source

Device on which tensor is resident.

dtype​

property dtype

source

DType of constituent elements in tensor.

element_size​

property element_size

source

Return the size of the element type in bytes.

from_dlpack()​

static from_dlpack(array, *, copy=None)

source

Create a buffer from an object implementing the dlpack protocol.

This usually does not result in a copy, and the producer of the object retains ownership of the underlying memory.

Parameters:

  • array (Any) – An object that implements the dlpack protocol.
  • copy (bool | None) – Whether to create a copy of the data. Defaults to None, which copies only when the producer requires it.

Returns:

A new buffer that views or copies the data.

Return type:

Buffer

from_numpy()​

static from_numpy(arr)

source

Creates a buffer from a provided numpy array on the host device.

The underlying data is not copied unless the array is noncontiguous. If it is, a contiguous copy will be returned.

Parameters:

arr (ndarray[tuple[Any, ...], dtype[Any]]) – The numpy array to wrap.

Returns:

A new buffer that views or copies the numpy array data.

Return type:

Buffer

inplace_copy_from()​

inplace_copy_from(src)

source

Copy the contents of another buffer into this one.

These buffers may be on different devices. Requires that both buffers are contiguous and have same size.

Parameters:

Return type:

None

is_contiguous​

property is_contiguous

source

Whether or not buffer is contiguously allocated in memory.

Returns false if the buffer is a non-contiguous slice.

Currently, we consider certain situations that are contiguous as non-contiguous for the purposes of our engine, such as when a buffer has negative steps.

is_host​

property is_host

source

Whether or not buffer is host-resident.

Returns false for GPU buffers, true for CPU buffers.

from max import driver
from max.dtype import DType

cpu_buffer = driver.Buffer(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU())

print(cpu_buffer.is_host)

item()​

item() → Any

source

Returns the scalar value at a given location.

Currently implemented only for zero-rank buffers. The return type is converted to a Python built-in type.

mmap()​

static mmap(filename, dtype, shape, mode='copyonwrite', offset=0)

source

Creates a memory-mapped buffer from a binary file on disk.

The argument semantics follow ~numpy.memmap.

Parameters:

  • filename (PathLike[str] | str) – Path of the binary file to map.
  • dtype (DType) – Element type of the mapped data.
  • shape (ShapeType | int) – Shape of the mapped buffer, either a tuple or an int.
  • mode (np._MemMapModeKind) – Memory-map mode, following ~numpy.memmap semantics.
  • offset (int) – Byte offset into the file where the mapping starts.

Returns:

A new buffer backed by the memory-mapped file.

Return type:

Buffer

num_elements​

property num_elements

source

Returns the number of elements in this buffer.

Rank-0 buffers have 1 element by convention.

pinned​

property pinned

source

Whether the allocation landed in the device’s host memory space. Ask usage for what was requested.

rank​

property rank

source

Buffer rank.

scalar()​

scalar(value: Any, dtype: max.dtype.DType, device: max.driver.Device | None = None) → max.driver.Buffer

source

Creates a scalar value of a given dtype and value.

If device is None (default), the buffer will be allocated on the CPU.

shape​

property shape

source

Shape of buffer.

stream​

property stream

source

Stream to which tensor is bound.

strides​

property strides

source

How far apart consecutive elements are, per dimension.

Counted in elements, not bytes.

Slicing does not move any data, so a slice keeps its parent’s strides. Narrow a 10-wide buffer to 6 columns and it still takes 10 elements to reach the next row:

parent = driver.Buffer(shape=[8, 10], dtype=DType.uint8)
rows = parent[:, :6]
rows.shape          # (8, 6)
rows.strides        # (10, 1) -- still 10 to the next row
rows.is_contiguous  # False

A buffer must be contiguous to be passed into model execution.

to()​

to(device: max.driver.Device) → max.driver.Buffer

source

Overloaded function.

Overload 1: to(device: Device) -> Buffer

Returns a buffer that’s guaranteed to be on the given device.

The buffer is only copied if the requested device is different from the device upon which the buffer is already resident.

Overload 2: to(devices: Sequence[Device]) -> list[Buffer]

Returns a list of buffers that are guaranteed to be on the given devices.

The buffers are only copied if the requested devices are different from the device upon which the buffer is already resident.

to_numpy()​

to_numpy()

source

Converts the buffer to a numpy array.

If the buffer is not on the host, a copy will be issued.

Parameters:

self (Buffer)

Return type:

ndarray[tuple[Any, …], dtype[Any]]

usage​

property usage

source

Allocation intent. Slices and views report their parent’s usage.

view()​

view(dtype, shape=None)

source

Return a new buffer with the given type and shape that shares the underlying memory.

If the shape is not given, it will be deduced if possible, or a ValueError is raised.

Parameters:

Return type:

Buffer

zeros()​

zeros(shape: collections.abc.Sequence[int], dtype: max.dtype.DType, device: max.driver.Device | None = None, usage: max.driver.Usage = Usage.DEFAULT) → max.driver.Buffer

source

Allocates a buffer with all elements initialized to zero.

Parameters:

  • shape (Sequence[int]) – The shape of the buffer.
  • dtype (DType) – The data type of the buffer.
  • device (Device, optional) – The device to allocate the buffer on. Defaults to None (CPU).
  • usage (Usage, optional) – Allocation intent, see Usage. Defaults to Usage.DEFAULT.

Returns:

A new buffer filled with zeros.

Return type:

Buffer