IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

DeviceContext

struct DeviceContext[target: CompilationTarget = get_gpu_target()]

Represents a single stream of execution on a particular accelerator (GPU).

A DeviceContext serves as the low-level interface to the accelerator inside a MAX custom operation and provides methods for allocating buffers on the device, copying data between host and device, and for compiling and running functions (also known as kernels) on the device.

The device context can be used as a context manager. For example:

from max.gpu.host import DeviceContext
from max.gpu import thread_idx

def kernel():
    print("hello from thread:", thread_idx.x, thread_idx.y, thread_idx.z)

with DeviceContext() as ctx:
    ctx.enqueue_function[kernel](grid_dim=1, block_dim=(2, 2, 2))
    ctx.synchronize()

Parameters​

Implemented traits​

AnyType, Copyable, Deinitable, ImplicitlyCopyable, Movable, RegisterPassable, _DispatchEnqueuer

comptime members​

CPU​

comptime CPU = DeviceContext[CompilationTarget()]

DeviceContext type for the current CPU target.

default_device_info​

comptime default_device_info = LegacyDeviceContext[target].default_device_info if use_legacy_backend() else get_gpu_info[CompilationTarget._arch()]()

GPUInfo object for the default accelerator.

ExecutionConfigType​

comptime ExecutionConfigType = LegacyDeviceContext[target].ExecutionConfigType if use_legacy_backend() else ClusterAttributeLaunchConfig if conforms_to(DefaultMaxPlugin if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected.HookFor[DeviceSpec.target[_HALDeviceContext[target].device_spec]()].HalHooks.ExecutionConfigType[DeviceRef(_HALDeviceContext[target].device_spec, Int(0), private=Tuple())], ClusterExecutionConfig) else GridBlockAttributeLaunchConfig if conforms_to(DefaultMaxPlugin if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected.HookFor[DeviceSpec.target[_HALDeviceContext[target].device_spec]()].HalHooks.ExecutionConfigType[DeviceRef(_HALDeviceContext[target].device_spec, Int(0), private=Tuple())], LaunchAttributeHolderExecutionConfig) else ClusterLaunchConfig if conforms_to(DefaultMaxPlugin if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected.HookFor[DeviceSpec.target[_HALDeviceContext[target].device_spec]()].HalHooks.ExecutionConfigType[DeviceRef(_HALDeviceContext[target].device_spec, Int(0), private=Tuple())], ClusterExecutionConfig) else DefaultMaxPlugin if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected.HookFor[DeviceSpec.target[_HALDeviceContext[target].device_spec]()].HalHooks.ExecutionConfigType[DeviceRef(_HALDeviceContext[target].device_spec, Int(0), private=Tuple())] if DefaultMaxPlugin if not _index_lt[PluginSelector._find[_mlir_value, 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[_mlir_value, 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[_mlir_value, 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[_mlir_value, 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[target, _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[target, _NoMaxPlugin].selected.HookFor[target].HalHooks.Launcher.launches_config and not conforms_to(DefaultMaxPlugin if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[DeviceSpec._mlir_target[_HALDeviceContext[target].device_spec](), 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[CompilationTarget(), _NoMaxPlugin].selected.HookFor[DeviceSpec.target[_HALDeviceContext[target].device_spec]()].HalHooks.ExecutionConfigType[DeviceRef(_HALDeviceContext[target].device_spec, Int(0), private=Tuple())], GridBlockExecutionConfig).__bool__() else GridBlockLaunchConfig

The launch configuration type with every launch capability this target supports.

On NVIDIA targets it carries launch attributes, and from sm_90 on also thread block clusters. When the plugin's own configuration isn't grid/block shaped and the plugin's launcher takes it, it's that configuration.

Generic code checks for a capability with conforms_to, for example conforms_to(DeviceContext[].ExecutionConfigType, ClusterExecutionConfig).

FunctionWrapper​

comptime FunctionWrapper = DefaultMaxPlugin if not _index_lt[PluginSelector._find[_mlir_value, 0, trap_if_missing=False](), 0]().__bool__() else [PluginSelector._find[_mlir_value, 0, trap_if_missing=False]()] if not _index_lt[PluginSelector._find[_mlir_value, 0, trap_if_missing=False](), 3]().__bool__() else CudaMaxPlugin, HipMaxPlugin, MetalMaxPlugin[PluginSelector._find[_mlir_value, 0, trap_if_missing=False]()] if Bool(identical(TargetPluginSelector[target, _NoMaxPlugin].selected, _NoMaxPlugin)) else TargetPluginSelector[target, _NoMaxPlugin].selected.HookFor[target].FunctionWrapper

Selects the function compile_function compiles for a user kernel.

NoneValue​

comptime NoneValue = Optional()

Convenience helper for constructing a default value for DeviceContext.Optional parameters. This guides type inference to the CPU target when the default argument value is used, so omitting the context means "run on the CPU".

Intended to be used like:

ctx: DeviceContext.Optional = DeviceContext.NoneValue,

Optional​

comptime Optional = Optional[DeviceContext[target]]

Convenience type alias for an Optional[DeviceContext[_]] type parameter.

This associated type worksaround Mojo's missing support for nested auto-parameterization. Mojo does not currently support argument types like:

ctx: Optional[DeviceContext[_]],

because _ unbound parameters can only be used in the top-level type of a argument type annotation.

However, using this alias, auto-parameterization works:

ctx: DeviceContext.Optional,

# equivalent, also works:
ctx: DeviceContext.Optional,

This alias may be removed once Mojo supports nested auto-parameterization.

Unknown​

comptime Unknown = DeviceContext[CompilationTarget()]

DeviceContext type whose compilation target is not statically known.

Methods​

__init__​

def __init__(out self, device_id: Int = Int(0), *, var api: String = DeviceContext._default_api())

Constructs a DeviceContext for the specified device.

This initializer creates a new device context for the specified accelerator device. The device context provides an interface for interacting with the GPU, including memory allocation, data transfer, and kernel execution.

Example:

from max.gpu.host import DeviceContext

# Create a context for the default GPU
var ctx = DeviceContext()

# Create a context for a specific GPU (device 1)
var ctx2 = DeviceContext(1)

Args:

  • ​device_id (Int): ID of the accelerator device. If not specified, uses the default accelerator (device 0).
  • ​api (String): Requested device API (for example, "cuda" or "hip"). Defaults to the device API of this context's target, which is "cpu" for a CPU target.

Raises:

If api does not match this context's target (a CPU target requires "cpu" and any other target requires a non-CPU API), if device initialization fails, or if the specified device is not available.

__eq__​

def __eq__(self, other: Self) -> Bool

Compares two contexts for identity.

Args:

  • ​other (Self): The context to compare with.

Returns:

Bool: Whether both contexts are the same.

__ne__​

def __ne__(self, other: Self) -> Bool

Compares two contexts for identity.

Args:

  • ​other (Self): The context to compare with.

Returns:

Bool: Whether the contexts differ.

cpu​

static def cpu() -> DeviceContext[_].CPU

Creates a CPU DeviceContext.

Returns:

DeviceContext[_].CPU: A new CPU DeviceContext.

assert_cpu_target​

def assert_cpu_target(var self) -> DeviceContext[_].CPU

Temporary function to assert and cast this DeviceContext to the CPU target.

This method will be removed once all uses have migrated to static CPU DeviceContext target types.

The check is performed at runtime on the context's API, since the static target of a plain DeviceContext is always the default accelerator, even for contexts created with api="cpu".

Returns:

DeviceContext[_].CPU: This device context cast to the CPU target.

assert_default_target​

def assert_default_target(self) -> DeviceContext

Temporary function to assert and cast this DeviceContext to the default target.

This method will be removed once all APIs fully support generic target parameters.

Constraints:

This DeviceContext must be using the default target, else a compile time assertion will fail.

Returns:

DeviceContext: This device context cast to the default target.

into_unknown_target​

def into_unknown_target(self) -> DeviceContext[_].Unknown

Erases this DeviceContext's static target.

For code that only uses the context's handle, so it neither needs nor should assume a target.

Returns:

DeviceContext[_].Unknown: This device context with an unknown target.

unwrap_legacy​

def unwrap_legacy(self) -> LegacyDeviceContext[target]

Returns the underlying legacy context.

For code that needs a legacy-only member. Only valid in a build whose backend is the legacy one.

Returns:

LegacyDeviceContext[target]: The legacy device context this wrapper holds.

unwrap_hal​

def unwrap_hal(self) -> _HALDeviceContext[target]

Returns the underlying HAL context.

For code that needs a HAL-only member. Only valid in a build whose backend is the HAL one.

Returns:

_HALDeviceContext[target]: The HAL device context this wrapper holds.

__enter__​

def __enter__(var self) -> Self

Enables the use of DeviceContext in a 'with' statement context manager.

This method allows DeviceContext to be used with Python-style context managers, which ensures proper resource management and cleanup when the context exits.

Example:

from max.gpu.host import DeviceContext

# Using DeviceContext as a context manager
with DeviceContext() as ctx:
    # Perform GPU operations
    # Resources are automatically released when exiting the block
    pass

Returns:

Self: The DeviceContext instance to be used within the context manager block.

synchronize​

def synchronize(self)

Synchronizes the device context, waiting for all enqueued operations to complete.

This method blocks until all previously enqueued work on the context's stream has finished.

Raises:

If the synchronization operation fails.

id​

def id(self) -> Int64

Returns the ID of the device this context is associated with.

Deprecated: Use DeviceContext.device().id instead.

Returns:

Int64: The device ID.

Raises:

If querying the device ID fails.

get_memory_info​

def get_memory_info(self) -> Tuple[UInt, UInt]

Returns the free and total memory size for this device.

This method queries the current state of device memory, providing information about how much memory is available and the total memory capacity of the device. This is useful for memory management and determining if there's enough space for planned operations.

Example:

from max.gpu.host import DeviceContext

var ctx = DeviceContext()
try:
    (free, total) = ctx.get_memory_info()
    print("Free memory:", free / (1024*1024), "MB")
    print("Total memory:", total / (1024*1024), "MB")
except:
    print("Failed to get memory information")

Returns:

Tuple[UInt, UInt]: A tuple of (free memory, total memory) in bytes.

Raises:

If there's an error retrieving the memory information, or the device can't measure its free memory.

get_attribute​

def get_attribute(self, attr: DeviceAttribute) -> Int

Returns the value of a device attribute.

Deprecated: Use DeviceContext.device().get_attribute() instead.

Args:

  • ​attr (DeviceAttribute): The device attribute to query.

Returns:

Int: The attribute's value.

Raises:

If querying the attribute fails.

compute_capability​

def compute_capability(self) -> Int

Returns the compute capability of this device.

Returns:

Int: The device's compute capability.

Raises:

If querying the compute capability fails.

set_as_current​

def set_as_current(self)

For use with libraries that require a specific GPU context to be active. Sets the current device to the one associated with this DeviceContext.

Example:

from max.gpu.host import DeviceContext
var ctx = DeviceContext(device_id=1)
ctx.set_as_current()

Raises:

If there's an error setting the current device.

api​

def api(self) -> String

Returns the name of the API used to program the device.

This method queries the underlying device context to determine which GPU programming API is being used for the current device. This information is useful for writing code that can adapt to different GPU architectures and programming models.

Possible values are:

  • "cuda": NVIDIA GPUs using CUDA.
  • "hip": AMD GPUs using HIP.
  • "metal": Apple GPUs using Metal.
  • "cpu": CPU.
  • "": No accelerator available.

Returns:

String: The name of the API.

name​

def name(self) -> String

Returns the device name, defined by the native device API.

Deprecated: Use DeviceContext.device().get_name() instead.

Returns:

String: A string containing the device name.

get_api_version​

def get_api_version(self) -> Int

Returns the API version associated with this device.

This method retrieves the version number of the GPU driver currently installed on the system for the device associated with this context. The version is returned as an integer that can be used to check compatibility with specific features or to troubleshoot driver-related issues.

Example:

from max.gpu.host import DeviceContext

with DeviceContext() as ctx:
    # Get the API version
    var api_version = ctx.get_api_version()
    print("GPU API version:", api_version)

Returns:

Int: An integer representing the driver version.

Raises:

If the driver version cannot be retrieved or if the device context is invalid.

arch_name​

def arch_name(self) -> String

Returns the architecture name of this device.

Deprecated: Use DeviceContext.device().get_arch() instead.

Returns:

String: The device architecture name.

Raises:

If querying the architecture fails.

is_compatible​

def is_compatible(self) -> Bool

Returns True if this device is compatible with MAX.

This method checks whether the current device is compatible with the Modular Accelerated Execution (MAX) runtime. It's useful for validating that the device can execute the compiled code before attempting operations.

Example:

from max.gpu.host import DeviceContext

var ctx = DeviceContext()
print("Device is compatible with MAX:", ctx.is_compatible())

Returns:

Bool: True if the device is compatible with MAX, False otherwise.

is_host_unified​

def is_host_unified(self) -> Bool

Returns whether this device shares memory with the host.

Deprecated: Use DeviceContext.device().is_host_unified() instead.

Returns:

Bool: True when device and host memory are unified.

Raises:

If the query fails.

is_graph_capturing​

def is_graph_capturing(self) -> Bool

Returns whether work enqueued on this context is only recorded.

Returns:

Bool: True while a graph capture is in progress. Always False on the HAL backend, which has no stream capture.

Raises:

If the query fails. The HAL backend never raises.

number_of_devices​

static def number_of_devices(*, var api: String = DeviceContext._default_api()) -> Int

Returns the number of devices available that support the specified API.

Args:

  • ​api (String): Requested device API (for example, "cuda" or "hip"). Defaults to the device API specified by the current target accelerator.

Returns:

Int: The number of devices, or 0 if no driver plugin is available or the loaded plugin does not support the requested API.

stream​

def stream(self) -> DeviceStream

Returns the context's default stream.

Returns:

DeviceStream: The default DeviceStream.

Raises:

If the stream cannot be created.

create_stream​

def create_stream(self, *, priority: Int = Int(0)) -> DeviceStream

Creates a new stream on this context's device.

Args:

  • ​priority (Int): Requested stream priority. The HAL queue API has no priority support, so the HAL backend accepts it for parity with the legacy backend and ignores it.

Returns:

DeviceStream: The new DeviceStream.

Raises:

If the stream cannot be created.

create_event​

def create_event(self) -> DeviceEvent

Creates a new event for synchronization between streams.

Example:

from max.gpu.host import DeviceContext

var ctx = DeviceContext()

var default_stream = ctx.stream()
var new_stream = ctx.create_stream()

# Create an event
var event = ctx.create_event()

# Record the event in the default stream
default_stream.record_event(event)

# new_stream waits for the event, then continues
new_stream.enqueue_wait_for(event)
default_stream.synchronize()

Returns:

DeviceEvent: A DeviceEvent that can be used for synchronization.

Raises:

If the event cannot be created.

allocate_completion_flag​

def allocate_completion_flag(self) -> CompletionFlag

Allocates a CompletionFlag, initialized to 0, on this device.

Returns:

CompletionFlag: The new flag.

Raises:

If the allocation fails, or if the device's plugin has no completion flags.

supports_flag_wait​

def supports_flag_wait(self) -> Bool

Returns whether this device supports waiting on a completion flag.

Returns:

Bool: True when flag waits are supported.

num_streams​

def num_streams(self) -> Int

Returns the number of streams available on this device context.

Returns:

Int: The number of streams available on this device context.

select_stream​

def select_stream(self, stream_id: Int) -> Self

Selects the stream that this context's work runs on.

Args:

  • ​stream_id (Int): The index of the stream to select.

Returns:

Self: A context bound to the selected stream.

Raises:

If the stream cannot be selected.

create_external_stream​

def create_external_stream(self, external_stream: Optional[Pointer[NoneType, origin]]) -> DeviceStream

Wraps a stream created outside of MAX.

Args:

Returns:

DeviceStream: A device stream backed by external_stream.

Raises:

If the stream cannot be wrapped.

stream_priority_range​

def stream_priority_range(self) -> StreamPriorityRange

Returns the range of stream priorities supported by this device context.

Returns:

StreamPriorityRange: A StreamPriorityRange containing the minimum and maximum stream priorities.

Raises:

If the query fails.

push_context​

def push_context(self) -> _DeviceContextScope[target]

Returns a context manager that makes this device's driver context current for the duration of a with block.

Example:

from max.gpu.host import DeviceContext

var ctx = DeviceContext(device_id=1)
# Ensure GPU 1's context is active for these operations.
with ctx.push_context():
    # All GPU operations here will use GPU 1's context.
    ...  # call external stateful APIs, such as cudnn.
# Previous context is automatically restored

Returns:

_DeviceContextScope[target]: A context manager that manages the driver context stack.

Raises:

If the driver contexts cannot be switched.

device​

def device(ref self) -> Device[target, origin_of(self)]

Returns a view of the device this context runs on.

Returns:

Device[target, origin_of(self)]: A Device that borrows this context.

enqueue_create_buffer​

def enqueue_create_buffer[dtype: DType](self, size: Int) -> DeviceBuffer[dtype]

Enqueues a buffer creation using the DeviceBuffer constructor.

For GPU devices, the space is allocated in the device's global memory.

Parameters:

  • ​dtype (DType): The data type to be stored in the allocated memory.

Args:

  • ​size (Int): The number of elements of type to allocate memory for.

Returns:

DeviceBuffer[dtype]: The allocated buffer.

Raises:

If the operation fails.

create_buffer_sync​

def create_buffer_sync[dtype: DType](self, size: Int) -> DeviceBuffer[dtype]

Allocates a device buffer and waits for the allocation to complete.

Parameters:

  • ​dtype (DType): The element dtype of the buffer.

Args:

  • ​size (Int): Number of elements to allocate.

Returns:

DeviceBuffer[dtype]: The allocated device buffer.

Raises:

If the allocation fails.

enqueue_create_host_buffer​

def enqueue_create_host_buffer[dtype: DType](self, size: Int) -> HostBuffer[dtype]

Enqueues the creation of a HostBuffer.

This function allocates memory on the host that is accessible by the device. The memory is page-locked (pinned) for efficient data transfer between host and device.

Pinned memory is guaranteed to remain resident in the host's RAM, not be paged/swapped out to disk. Memory allocated normally (for example, using alloc()) is pageable—individual pages of memory can be moved to secondary storage (disk/SSD) when main memory fills up.

Using pinned memory allows devices to make fast transfers between host memory and device memory, because they can use direct memory access (DMA) to transfer data without relying on the CPU.

Allocating too much pinned memory can cause performance issues, since it reduces the amount of memory available for other processes.

Example:

from max.gpu.host import DeviceContext

with DeviceContext() as ctx:
    # Allocate host memory accessible by the device
    var host_buffer = ctx.enqueue_create_host_buffer[.float32](1024)

    # Use the host buffer for device operations
    # ...

Parameters:

  • ​dtype (DType): The data type to be stored in the allocated memory.

Args:

  • ​size (Int): The number of elements of type to allocate memory for.

Returns:

HostBuffer[dtype]: A HostBuffer object that wraps the allocated host memory.

Raises:

If memory allocation fails or if the device context is invalid.

wrap_host_memory​

def wrap_host_memory[origin: MutOrigin, //, dtype: DType](self, host_ptr: Pointer[Scalar[dtype], origin], size: Int) -> DeviceBuffer[dtype]

Makes a range of the caller's host memory accessible to this device.

The returned buffer grants access, not ownership. host_ptr stays the caller's to keep mapped and to free, and dropping the buffer undoes only the access.

Address the range through the returned buffer, not through host_ptr: only on CUDA are the two the same address.

Parameters:

  • ​origin (MutOrigin): The origin of host_ptr.
  • ​dtype (DType): The element dtype of the range.

Args:

Returns:

DeviceBuffer[dtype]: A device buffer addressing the same memory.

Raises:

If the range cannot be registered with the driver.

enqueue_copy​

def enqueue_copy[dtype: DType](self, dst_ptr: Pointer[Scalar[dtype]], src_ptr: Pointer[Scalar[dtype]], size: Int)

Enqueues an async copy of size elements from a device pointer to another device pointer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst_buf: DeviceBuffer[dtype], src_ptr: Pointer[Scalar[dtype]])

Enqueues an async copy from the host to the provided device buffer. The number of bytes copied is determined by the size of the device buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (DeviceBuffer[dtype]): Device buffer to copy to.
  • ​src_ptr (Pointer[Scalar[dtype]]): Host pointer to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst_buf: HostBuffer[dtype], src_ptr: Pointer[Scalar[dtype]])

Enqueues an async copy from the host to the provided device buffer. The number of bytes copied is determined by the size of the device buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (HostBuffer[dtype]): Device buffer to copy to.
  • ​src_ptr (Pointer[Scalar[dtype]]): Host pointer to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst_ptr: Pointer[Scalar[dtype]], src_buf: DeviceBuffer[dtype])

Enqueues an async copy from the device to the host. The number of bytes copied is determined by the size of the device buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_ptr (Pointer[Scalar[dtype]]): Host pointer to copy to.
  • ​src_buf (DeviceBuffer[dtype]): Device buffer to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst_buf: DeviceBuffer[dtype], src_buf: DeviceBuffer[dtype])

Enqueues an async copy from one device buffer to another. The amount of data transferred is determined by the size of the destination buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (DeviceBuffer[dtype]): Device buffer to copy to.
  • ​src_buf (DeviceBuffer[dtype]): Device buffer to copy from. Must be at least as large as dst.

Raises:

If the operation fails, or if dst_buf is not on this context's device.

def enqueue_copy[dtype: DType](self, dst_buf: DeviceBuffer[dtype], src_buf: HostBuffer[dtype])

Enqueues an async copy from one device buffer to another. The amount of data transferred is determined by the size of the destination buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (DeviceBuffer[dtype]): Device buffer to copy to.
  • ​src_buf (HostBuffer[dtype]): Device buffer to copy from. Must be at least as large as dst.

Raises:

If the operation fails, or if src_buf is smaller than dst_buf.

def enqueue_copy[dtype: DType](self, dst_buf: HostBuffer[dtype], src_buf: DeviceBuffer[dtype])

Enqueues an async copy from one device buffer to another. The amount of data transferred is determined by the size of the destination buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (HostBuffer[dtype]): Device buffer to copy to.
  • ​src_buf (DeviceBuffer[dtype]): Device buffer to copy from. Must be at least as large as dst.

Raises:

If the operation fails, or if src_buf is smaller than dst_buf.

def enqueue_copy[dtype: DType](self, dst_buf: HostBuffer[dtype], src_buf: HostBuffer[dtype])

Enqueues an async copy from one device buffer to another. The amount of data transferred is determined by the size of the destination buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (HostBuffer[dtype]): Device buffer to copy to.
  • ​src_buf (HostBuffer[dtype]): Device buffer to copy from. Must be at least as large as dst.

Raises:

If the operation fails, or if src_buf is smaller than dst_buf.

def enqueue_copy[dtype: DType](self, dst_ptr: Pointer[Scalar[dtype]], src_buf: HostBuffer[dtype])

Enqueues an async copy from the device to the host. The number of bytes copied is determined by the size of the device buffer.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_ptr (Pointer[Scalar[dtype]]): Host pointer to copy to.
  • ​src_buf (HostBuffer[dtype]): Device buffer to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst_buf: DeviceBuffer[dtype], src: Span[Scalar[dtype]])

Enqueues an async copy from a host Span to a device buffer.

The number of bytes copied is determined by the size of the device buffer. The span must contain at least as many elements as the destination buffer; this invariant is checked via debug_assert.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (DeviceBuffer[dtype]): Device buffer to copy to.
  • ​src (Span[Scalar[dtype]]): Host span to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst_buf: HostBuffer[dtype], src: Span[Scalar[dtype]])

Enqueues an async copy from a host Span to a pinned host buffer.

The number of bytes copied is determined by the size of the device buffer. The span must contain at least as many elements as the destination buffer; this invariant is checked via debug_assert.

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (HostBuffer[dtype]): Device buffer to copy to.
  • ​src (Span[Scalar[dtype]]): Host span to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst: Span[Scalar[dtype]], src_buf: DeviceBuffer[dtype])

Enqueues an async copy from a device buffer to a host Span.

The number of bytes copied is determined by the size of the device buffer. The span must contain at least as many elements as the source buffer; this invariant is checked via debug_assert (debug builds only).

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst (Span[Scalar[dtype]]): Host span to copy to.
  • ​src_buf (DeviceBuffer[dtype]): Device buffer to copy from.

Raises:

If the operation fails.

def enqueue_copy[dtype: DType](self, dst: Span[Scalar[dtype]], src_buf: HostBuffer[dtype])

Enqueues an async copy from a host buffer to a host Span.

The number of bytes copied is determined by the size of the source buffer. The span must contain at least as many elements as the source buffer; this invariant is checked via debug_assert (debug builds only).

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst (Span[Scalar[dtype]]): Host span to copy to.
  • ​src_buf (HostBuffer[dtype]): Host buffer to copy from.

Raises:

If the operation fails.

enqueue_copy_no_cross_stream_sync​

def enqueue_copy_no_cross_stream_sync[dtype: DType](self, dst_buf: DeviceBuffer[dtype], src_buf: DeviceBuffer[dtype])

Enqueues a device-to-device copy without cross-stream synchronization.

This behaves like enqueue_copy for two device buffers, except that when the source and destination are on different streams the driver does not insert its own cross-stream synchronization; the caller is responsible for ordering (for example via explicit device-wait ops).

Parameters:

  • ​dtype (DType): Type of the data being copied.

Args:

  • ​dst_buf (DeviceBuffer[dtype]): Destination device buffer.
  • ​src_buf (DeviceBuffer[dtype]): Source device buffer.

Raises:

If the operation fails.

enqueue_memset​

def enqueue_memset[dtype: DType](self, dst: DeviceBuffer[dtype], val: Scalar[dtype])

Enqueues an async memset operation, setting all of the elements in the destination device buffer to the specified value.

Parameters:

  • ​dtype (DType): Type of the data stored in the buffer.

Args:

  • ​dst (DeviceBuffer[dtype]): Destination buffer.
  • ​val (Scalar[dtype]): Value to set all elements of dst to.

Raises:

If the operation fails.

def enqueue_memset[dtype: DType](self, dst: HostBuffer[dtype], val: Scalar[dtype])

Enqueues an async memset operation, setting all of the elements in the destination host buffer to the specified value.

Parameters:

  • ​dtype (DType): Type of the data stored in the buffer.

Args:

  • ​dst (HostBuffer[dtype]): Destination buffer.
  • ​val (Scalar[dtype]): Value to set all elements of dst to.

Raises:

If the operation fails.

enqueue_cpu_range​

def enqueue_cpu_range[FuncType: def(Int) -> None & ImplicitlyCopyable](self, func: FuncType, count: Int)

Runs a function closure over a 1D range on a CPU DeviceContext.

The function is called as func(i) for each i in range(count). Instances run in parallel on the AsyncRT worker threads, but it is not guaranteed that all instances run at the same time.

The HAL backend has no host-function enqueue plugin entry, so the range runs synchronously there: it returns when every instance is done, and synchronize() is then a no-op for this work.

Parameters:

Args:

  • ​func (FuncType): The function closure to execute.
  • ​count (Int): The number of instances of the function to run.

Raises:

If self is not a CPU DeviceContext.

enqueue_cpu_function​

def enqueue_cpu_function[FuncType: def() -> None & ImplicitlyCopyable](self, func: FuncType)

Enqueues a function for execution on CPU.

Parameters:

Args:

  • ​func (FuncType): The function to execute.

Raises:

If the operation fails. If self is not a CPU DeviceContext.

compile_function​

def compile_function[declared_arg_types: TypeList[declared_arg_types.values], //, func: def(*args: *declared_arg_types) thin -> None, *, compile_options: StaticString = CompilationTarget.default_compile_options(), link_options: StaticString = "", dump_asm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, dump_llvm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _dump_sass: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _ptxas_info_verbose: Bool = False](self, *, func_attribute: OptionalReg[FuncAttribute] = None, out result: DeviceFunction[comptime[LITImmOrigin, ImmOrigin] #kgen.func.symbol<_"def[KGENParamList[::AnyType], ::TypeList[::AnyType, $0], LITImmOrigin, ::Origin[False, $2]](*args: *$1) thin -> None|def[LITImmOrigin, ::Origin[False, $0]](*args: *$1|1) thin -> None|9eda2ad4f38df052[KGENParamList[::AnyType], ::TypeList[::AnyType, $0], LITImmOrigin, ::Origin[False, $2], def[LITImmOrigin, ::Origin[False, $0]](*args: *$1|1) thin -> None](*$0)"<:param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x", :!lit.struct<_std::_builtin::_variadics::_TypeList<:meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x">> declared_arg_types, :origin<false> *(0, 0), :!lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0)>> *(0, 1), :!lit.generator<<origin<false>, !lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0)>>>[1]("args": !lit.ref<!lit.struct<_std::_builtin::_variadics::_VariadicPack<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0), :!lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0)>> *(0, 1), :meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x", :!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :!lit.struct<_std::_builtin::_variadics::_TypeList<:meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x">> declared_arg_types>>, imm *[0, 0]> imm_mem|pack_vararg) -> !kgen.none> rebind(:!lit.generator<<?, "args.origin._mlir_origin3x": origin, "args.origin3x1": !lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0)>>>[1]("args": !lit.ref<!lit.struct<_std::_builtin::_variadics::_VariadicPack<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0), :!lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0, 0)>> *(0, 1), :meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values2x", :!lit.struct<_std::_builtin::_bool::_Bool> {:scalar false}, :!lit.struct<_std::_builtin::_variadics::_TypeList<:meta> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x">> declared_arg_types>>, imm *[0, 0]> imm_mem|pack_vararg) -> !kgen.none> func)>>, Self.FunctionWrapper, target=target, compile_options=compile_options, link_options=link_options, _ptxas_info_verbose=_ptxas_info_verbose]) ``

Compiles the provided function for execution on this device.

Parameters:

  • ​declared_arg_types (TypeList[values]): Types of the arguments of the kernel.
  • ​func (def(*args: *declared_arg_types) thin -> None): The function to compile.
  • ​compile_options (StaticString): The compile options for the function.
  • ​link_options (StaticString): The link options for the function.
  • ​dump_asm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the compiled assembly, pass True, or a file path to dump to, or a function returning a file path.
  • ​dump_llvm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the generated LLVM code, pass True, or a file path to dump to, or a function returning a file path.
  • ​_dump_sass (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed. Pass True, or a file path to dump to, or a function returning a file path.
  • ​_ptxas_info_verbose (Bool): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed. Changes dump_asm to output verbose PTX assembly (default False).

Args:

Returns:

DeviceFunction[comptime[LITImmOrigin, ImmOrigin] #kgen.func.symbol<_"def[KGENParamList[::AnyType], ::TypeList[::AnyType, $0], LITImmOrigin, ::Origin[False, $2]](*args: *$1) thin -> None|def[LITImmOrigin, ::Origin[False, $0]](*args: *$1|1) thin -> None|9eda2ad4f38df052[KGENParamList[::AnyType],::TypeList[::AnyType, $0],LITImmOrigin,::Origin[False, $2],def[LITImmOrigin, ::Origin[False, $0]](*args: *$1|1) thin -> None](*$0)"<:param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x", :!lit.struct<_std::_builtin::_variadics::_TypeList<:meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x">> declared_arg_types, :origin<false> *(0,0), :!lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0)>> *(0,1), :!lit.generator<<origin<false>, !lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0)>>>[1]("args": !lit.ref<!lit.struct<_std::_builtin::_variadics::_VariadicPack<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0), :!lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0)>> *(0,1), :meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x", :!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :!lit.struct<_std::_builtin::_variadics::_TypeList<:meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x">> declared_arg_types>>, imm *[0,0]> imm_mem|pack_vararg) -> !kgen.none> rebind(:!lit.generator<<?, "args.origin._mlir_origin3x": origin, "args.origin3x1": !lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0)>>>[1]("args": !lit.ref<!lit.struct<_std::_builtin::_variadics::_VariadicPack<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0), :!lit.struct<_std::_origin::___init__::_Origin<:!lit.struct<_std::_builtin::_bool::_Bool> {:scalar<bool> false}, :origin<false> *(0,0)>> *(0,1), :meta<!lit.trait<_std::_traits::_anytype::_AnyType>> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values2x", :!lit.struct<_std::_builtin::_bool::_Bool> {:scalar false}, :!lit.struct<_std::_builtin::_variadics::_TypeList<:meta> trait<_std::_traits::_anytype::_AnyType>, :param_list<trait<_std::_traits::_anytype::_AnyType>> *"declared_arg_types.values`2x">> declared_arg_types>>, imm *[0,0]> imm_mem|pack_vararg) -> !kgen.none> func)>>, Self.FunctionWrapper, target=target, compile_options=compile_options, link_options=link_options, _ptxas_info_verbose=_ptxas_info_verbose]``: The compiled function.

Raises:

If the compilation fails.

def compile_function[ClosureType: def() -> None & RegisterPassable](self) -> LegacyDeviceClosureFunction[ClosureType, target]

Compiles a closure as a kernel that takes no arguments.

Launch the result with enqueue_function(kernel, closure, ...). Only targets whose MAX plugin handles closure kernels support this, and the HAL backend has no closure kernels.

Parameters:

  • ​ClosureType (def() -> None & RegisterPassable): The closure's type, such as type_of(closure).

Returns:

LegacyDeviceClosureFunction[ClosureType, target]: The compiled kernel.

Raises:

If compilation or loading fails.

enqueue_function​

def enqueue_function[ClosureType: def() -> None & RegisterPassable](self, f: LegacyDeviceClosureFunction[ClosureType, target], closure: ClosureType, *, grid_dim: Dim, block_dim: Dim, location: Optional[SourceLocation] = None)

Enqueues a closure kernel compiled by compile_function[ClosureType].

Parameters:

Args:

  • ​f (LegacyDeviceClosureFunction[ClosureType, target]): The compiled kernel.
  • ​closure (ClosureType): The closure whose captures to launch with.
  • ​grid_dim (Dim): The grid dimensions.
  • ​block_dim (Dim): The block dimensions.
  • ​location (Optional[SourceLocation]): Source location for the function call.

Raises:

If the launch fails.

def enqueue_function[*Ts: AnyType](self, f: DeviceExternalFunction[target=f.target], *args: *Ts.values, *, grid_dim: Dim, block_dim: Dim, cluster_dim: OptionalReg[Dim] = None, shared_mem_bytes: OptionalReg[Int] = None, var attributes: List[LaunchAttribute] = List(__list_literal__=NoneType(None)), var constant_memory: List[ConstantMemoryMapping] = List(__list_literal__=NoneType(None)), location: Optional[SourceLocation] = None)

Enqueues a loaded external function for execution on this device.

Parameters:

  • ​*Ts (AnyType): The types of the arguments.

Args:

  • ​f (DeviceExternalFunction[target=f.target]): The loaded function to launch.
  • ​*args (*Ts.values): The arguments to pass to the function.
  • ​grid_dim (Dim): The grid dimensions.
  • ​block_dim (Dim): The block dimensions.
  • ​cluster_dim (OptionalReg[Dim]): The cluster dimensions.
  • ​shared_mem_bytes (OptionalReg[Int]): The dynamic shared memory in bytes.
  • ​attributes (List[LaunchAttribute]): The launch attributes.
  • ​constant_memory (List[ConstantMemoryMapping]): The constant memory mappings.
  • ​location (Optional[SourceLocation]): The source location of the call.

Raises:

If the launch fails.

def enqueue_function[*Ts: DevicePassable](self, f: DeviceFunction[target=f.target, compile_options=f.compile_options, link_options=f.link_options, _ptxas_info_verbose=f._ptxas_info_verbose], *args: *Ts.values, *, grid_dim: Dim, block_dim: Dim, cluster_dim: OptionalReg[Dim] = None, shared_mem_bytes: OptionalReg[Int] = None, var attributes: List[LaunchAttribute] = List(__list_literal__=NoneType(None)), var constant_memory: List[ConstantMemoryMapping] = List(__list_literal__=NoneType(None)), location: Optional[SourceLocation] = None)

Enqueues a compiled function for execution on this device.

The function must have been compiled with type checking enabled (via compile_function), which verifies at compile time that the argument types match the declared types.

from max.gpu.host import DeviceContext

def kernel(x: Int):
    print("Value:", x)

with DeviceContext() as ctx:
    var compiled_func = ctx.compile_function[kernel]()
    ctx.enqueue_function(compiled_func, 42, grid_dim=1, block_dim=1)
    ctx.synchronize()

Parameters:

Args:

  • ​f (DeviceFunction[target=f.target, compile_options=f.compile_options, link_options=f.link_options, _ptxas_info_verbose=f._ptxas_info_verbose]): The compiled function to launch.
  • ​*args (*Ts.values): The arguments to pass to the function.
  • ​grid_dim (Dim): The grid dimensions.
  • ​block_dim (Dim): The block dimensions.
  • ​cluster_dim (OptionalReg[Dim]): The cluster dimensions.
  • ​shared_mem_bytes (OptionalReg[Int]): The dynamic shared memory in bytes.
  • ​attributes (List[LaunchAttribute]): The launch attributes.
  • ​constant_memory (List[ConstantMemoryMapping]): The constant memory mappings.
  • ​location (Optional[SourceLocation]): The source location of the call.

Raises:

If the launch fails.

def enqueue_function[declared_arg_types: TypeList[declared_arg_types.values], HostArgs: TypeList[HostArgs.values], //, func: def(*args: *declared_arg_types) thin -> None, *encoded_types: DevicePassable, *, link_options: StaticString = "", dump_asm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, dump_llvm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _dump_sass: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _ptxas_info_verbose: Bool = False](self, *encoded_args: *encoded_types.values, *, host_args: Tuple[*HostArgs.values] = Tuple(), grid_dim: Dim, block_dim: Dim, cluster_dim: OptionalReg[Dim] = None, shared_mem_bytes: OptionalReg[Int] = None, var attributes: List[LaunchAttribute] = List(__list_literal__=NoneType(None)), var constant_memory: List[ConstantMemoryMapping] = List(__list_literal__=NoneType(None)), func_attribute: OptionalReg[FuncAttribute] = None, location: Optional[SourceLocation] = None)

Compiles and enqueues a kernel for execution on this device.

Put capturing closures in host_args. The named kernel is compiled against those host callables; encoding them would rewrite captures to a synthesized device_type the kernel cannot read. Kernel parameters must be encoded args first, then the host arguments.

Build host_args inline in the call with a tuple:

ctx.enqueue_function[kernel](
    buffer,
    host_args=(epilogue_fn,),
    grid_dim=1,
    block_dim=1,
)

Parameters:

  • ​declared_arg_types (TypeList[values]): Types of the arguments of the kernel.
  • ​HostArgs (TypeList[values]): Types of the host-only arguments.
  • ​func (def(*args: *declared_arg_types) thin -> None): The function to compile and launch.
  • ​*encoded_types (DevicePassable): The types of the device arguments.
  • ​link_options (StaticString): The link options for the function.
  • ​dump_asm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the compiled assembly, pass True, or a file path to dump to, or a function returning a file path.
  • ​dump_llvm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the generated LLVM code, pass True, or a file path to dump to, or a function returning a file path.
  • ​_dump_sass (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed.
  • ​_ptxas_info_verbose (Bool): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed.

Args:

Raises:

If the compilation or the launch fails.

def enqueue_function[FuncType: def() -> None & DevicePassable, //, dump_asm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, dump_llvm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _dump_sass: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _ptxas_info_verbose: Bool = False](self, func: FuncType, grid_dim: Dim, block_dim: Dim, cluster_dim: OptionalReg[Dim] = None, shared_mem_bytes: OptionalReg[Int] = None, var attributes: List[LaunchAttribute] = List(__list_literal__=NoneType(None)), var constant_memory: List[ConstantMemoryMapping] = List(__list_literal__=NoneType(None)), func_attribute: OptionalReg[FuncAttribute] = None, location: Optional[SourceLocation] = None)

Compiles and enqueues a capturing kernel whose captures are device passable.

Parameters:

  • ​FuncType (def() -> None & DevicePassable): The type of the capturing kernel.
  • ​dump_asm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the compiled assembly, pass True, or a file path to dump to, or a function returning a file path.
  • ​dump_llvm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the generated LLVM code, pass True, or a file path to dump to, or a function returning a file path.
  • ​_dump_sass (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed.
  • ​_ptxas_info_verbose (Bool): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed.

Args:

Raises:

If the compilation or the launch fails.

def enqueue_function[FuncType: def() -> None & RegisterPassable & ImplicitlyCopyable, //, dump_asm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, dump_llvm: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _dump_sass: Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path] = False, _ptxas_info_verbose: Bool = False](self, func: FuncType, grid_dim: Dim, block_dim: Dim, cluster_dim: OptionalReg[Dim] = None, shared_mem_bytes: OptionalReg[Int] = None, var attributes: List[LaunchAttribute] = List(__list_literal__=NoneType(None)), var constant_memory: List[ConstantMemoryMapping] = List(__list_literal__=NoneType(None)), func_attribute: OptionalReg[FuncAttribute] = None, location: Optional[SourceLocation] = None) where not conforms_to(FuncType, DevicePassable).__bool__()

Compiles and enqueues a capturing kernel whose captures are plain register-passable values.

Parameters:

  • ​FuncType (def() -> None & RegisterPassable & ImplicitlyCopyable): The type of the capturing kernel.
  • ​dump_asm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the compiled assembly, pass True, or a file path to dump to, or a function returning a file path.
  • ​dump_llvm (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): To dump the generated LLVM code, pass True, or a file path to dump to, or a function returning a file path.
  • ​_dump_sass (Variant[Bool, Path, StringSpan[ImmStaticOrigin], def() thin -> Path]): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed.
  • ​_ptxas_info_verbose (Bool): Only runs on NVIDIA targets, and requires CUDA toolkit to be installed.

Args:

Raises:

If the compilation or the launch fails.

def enqueue_function[declared_arg_types: TypeList[declared_arg_types.values], ConfigType: ExecutionConfig, HostArgs: TypeList[HostArgs.values], //, func: def(*args: *declared_arg_types) thin -> None, *encoded_types: DevicePassable](self, config: ConfigType, *encoded_args: *encoded_types.values, *, host_args: Tuple[*HostArgs.values] = Tuple(), location: Optional[SourceLocation] = None)

Compiles and enqueues a kernel with an execution configuration.

Put capturing closures in host_args, after the encoded arguments, as for the keyword overload.

You can pass the function directly to enqueue_function without compiling it first:

from max.gpu.host import DeviceContext
from max.driver._hal.execution_config import GPUExecutionConfiguration

def kernel():
    print("hello from the GPU")

with DeviceContext() as ctx:
    ctx.enqueue_function[kernel](GPUExecutionConfiguration(grid_dim=Dim(1, 1, 1), block_dim=Dim(1, 1, 1)))
    ctx.synchronize()

Parameters:

  • ​declared_arg_types (TypeList[values]): Types of the arguments of the kernel.
  • ​ConfigType (ExecutionConfig): The type of the execution configuration.
  • ​HostArgs (TypeList[values]): Types of the host-only arguments.
  • ​func (def(*args: *declared_arg_types) thin -> None): The function to compile and launch.
  • ​*encoded_types (DevicePassable): The types of the device arguments.

Args:

  • ​config (ConfigType): The execution configuration of the launch.
  • ​*encoded_args (*encoded_types.values): The device arguments to pass to the kernel.
  • ​host_args (Tuple[*HostArgs.values]): The host-only arguments to pass to the kernel.
  • ​location (Optional[SourceLocation]): The source location of the call.

Raises:

If the compilation or the launch fails.

def enqueue_function[ExecutionConfigType: ExecutionConfig, //, *Ts: DevicePassable](self, execution_config: ExecutionConfigType, f: DeviceFunction[target=f.target, compile_options=f.compile_options, link_options=f.link_options, _ptxas_info_verbose=f._ptxas_info_verbose], *args: *Ts.values, *, location: Optional[SourceLocation] = None)

Enqueues a pre-compiled checked function for execution on this device.

Only available on the HAL DeviceContext.

This overload requires a DeviceFunction that was compiled with type checking enabled (via compile_function). The function will verify that the argument types match the declared types at compile time.

from max.gpu.host import DeviceContext

def kernel(x: Int):
    print("Value:", x)

with DeviceContext() as ctx:
    var compiled_func = ctx.compile_function[kernel]()
    var cfg = GPUExecutionConfiguration(grid_dim=Dim(1, 1, 1), block_dim=Dim(1, 1, 1))
    ctx.enqueue_function(cfg, compiled_func, 42)
    ctx.synchronize()

Parameters:

Args:

  • ​execution_config (ExecutionConfigType): The execution configuration specifying device specific kernel launch arguments.
  • ​f (DeviceFunction[target=f.target, compile_options=f.compile_options, link_options=f.link_options, _ptxas_info_verbose=f._ptxas_info_verbose]): The compiled function to execute.
  • ​*args (*Ts.values): Arguments to pass to the function.
  • ​location (Optional[SourceLocation]): Source location for the function call.

Raises:

If the operation fails.

def enqueue_function[FuncType: def() -> None & RegisterPassable & ImplicitlyCopyable, ExecutionConfigType: ExecutionConfig, //](self, execution_config: ExecutionConfigType, func: FuncType, location: Optional[SourceLocation] = None)

Compiles and enqueues a capturing kernel for execution on this device.

Only available on the HAL DeviceContext.

This overload is for kernels that capture variables from their enclosing scope. The capturing annotation on the signature function indicates that the kernel can access variables from the surrounding context.

Parameters:

Args:

  • ​execution_config (ExecutionConfigType): The execution configuration specifying device specific kernel launch arguments.
  • ​func (FuncType): The capturing kernel function to compile and launch.
  • ​location (Optional[SourceLocation]): Source location for the function call.

Raises:

If the operation fails.

load_function​

def load_function[ret_type: AnyType, declared_arg_types: TypeList[declared_arg_types.values], //, func: def(*args: *declared_arg_types) thin -> ret_type](self, *, var function_name: String, var asm: String, func_attribute: OptionalReg[FuncAttribute] = None, out result: DeviceExternalFunction[target=target])

Loads a function from precompiled assembly.

This loads an external GPU function from the provided assembly code (PTX/SASS) rather than compiling it from Mojo source. This is useful for integrating with existing CUDA/HIP code or for using specialized assembly optimizations.

Parameters:

  • ​ret_type (AnyType): The return type of the function.
  • ​declared_arg_types (TypeList[values]): Types of the arguments of the function.
  • ​func (def(*args: *declared_arg_types) thin -> ret_type): The function signature.

Args:

  • ​function_name (String): The name of the function in the assembly.
  • ​asm (String): The assembly containing the function.
  • ​func_attribute (OptionalReg[FuncAttribute]): An attribute to use when loading the function. The HAL backend ignores it.

Returns:

DeviceExternalFunction[target=target]: The loaded function.

Raises:

If the load fails.

execution_time​

def execution_time[FuncType: def(Self) raises -> None](self, func: FuncType, num_iters: Int) -> Int

Measures the execution time, in nanoseconds, of func(self) run num_iters times (host monotonic clock, bracketed by synchronize()).

It runs the function for the specified number of iterations and returns the total elapsed time in nanoseconds.

Parameters:

  • ​FuncType (def(Self) raises -> None): The body function type.

Args:

  • ​func (FuncType): The closure carrying the captured state of the body function.
  • ​num_iters (Int): The number of iterations to run the function.

Returns:

Int: The total elapsed time in nanoseconds for all iterations.

Raises:

If the timing fails.

def execution_time[FuncType: def() raises -> None](self, func: FuncType, num_iters: Int) -> Int

Measures the execution time of a function over multiple iterations.

This overload is for a function that doesn't take the DeviceContext.

Parameters:

  • ​FuncType (def() raises -> None): The body function type.

Args:

  • ​func (FuncType): The closure carrying the captured state of the body function.
  • ​num_iters (Int): The number of iterations to run the function.

Returns:

Int: The total elapsed time in nanoseconds for all iterations.

Raises:

If the timing fails.

execution_time_iter​

def execution_time_iter[FuncType: def(Self, Int) raises -> None](self, func: FuncType, num_iters: Int) -> Int

Measures the execution time, in nanoseconds, of func(self, i) for i in range(num_iters) (host monotonic clock, bracketed by synchronize()).

Parameters:

  • ​FuncType (def(Self, Int) raises -> None): The body function type.

Args:

  • ​func (FuncType): The closure carrying the captured state of the body function.
  • ​num_iters (Int): The number of iterations to run the function.

Returns:

Int: The total elapsed time in nanoseconds for all iterations.

Raises:

If the timing fails.

can_access​

def can_access(self, peer: DeviceContext) -> Bool

Returns whether this context can access peer's memory.

Deprecated: Use DeviceContext.device().can_access_peer(peer.device()) instead.

Args:

  • ​peer (DeviceContext): The peer context.

Returns:

Bool: True if peer access is available.

Raises:

If the query fails.

enable_peer_access​

def enable_peer_access(self, peer: DeviceContext)

Enables direct memory access to the peer device.

Args:

  • ​peer (DeviceContext): The peer device context whose memory becomes accessible.

Raises:

If enabling peer access fails: the build or peer's driver lacks peer access, or the plugin refuses the pair.

enable_all_peer_access​

static def enable_all_peer_access()

Enables peer-to-peer access between all accelerators, both ways.

A silent no-op without peer access or with one accelerator, as on the legacy backend; collective setup warns on a raise.

Raises:

On the first pair the plugin refuses.

all_peer_access_enabled​

static def all_peer_access_enabled() -> Bool

Returns whether peer access is enabled between every pair of devices.

Returns:

Bool: True when all peer-access pairs are enabled.

Raises:

If the query fails.

supports_multicast​

def supports_multicast(self) -> Bool

Returns True if this device supports multicast memory mappings.

Returns:

Bool: True when multicast is available. Always False on the HAL backend, where multicast is not implemented.

Raises:

If the query fails. The HAL backend never raises.

enqueue_wait_for​

def enqueue_wait_for(self, other: DeviceContext)

Enqueues a wait for all work currently pending on other.

This creates a dependency between two device contexts: operations in the current context will not begin execution until all previously enqueued operations in the other context have completed. This is useful for synchronizing work across multiple devices or streams.

Example:

from max.gpu.host import DeviceContext

# Create two device contexts
var ctx1 = DeviceContext(0)  # First GPU
var ctx2 = DeviceContext(1)  # Second GPU

# Enqueue operations on ctx1
# ...

# Make ctx2 wait for ctx1 to complete before proceeding
ctx2.enqueue_wait_for(ctx1)

# Enqueue operations on ctx2 that depend on ctx1's completion
# ...

Args:

  • ​other (DeviceContext): The device context whose operations must complete before operations in this context can proceed.

Raises:

If there's an error enqueuing the wait operation or if the operation is not supported by the underlying device API.

Was this page helpful?