IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

dispatch_local

dispatch_local()​

max.nn.ep.dispatch_local(config, buffers, tokens, ids, *, input_scales=None, layer_idx=None)

source

Sends each token to its experts, with one fused kernel per device.

With config.use_allreduce, each device routes its own tokens to its own experts only, and sends nothing to the other devices. Each device gets its own copy of the grouped matmul bounds, as in dispatch_finish().

Parameters:

Returns:

The tokens each device received, and the route that combine_local() sends them back by.

Raises:

ValueError – If check_supported() rejects config, or if config counts expert load without layer_idx or without EPBuffers.expert_load.

Return type:

tuple[Grouped, Route]