For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python function
dispatch_local
dispatch_local()
max.nn.ep.dispatch_local(config, buffers, tokens, ids, *, input_scales=None, layer_idx=None)
Sends each token to its experts, with one fused kernel per device.
With config.use_allreduce, each device routes its own tokens to its
own experts only, and sends nothing to the other devices. Each device
gets its own copy of the grouped matmul bounds, as in
dispatch_finish().
-
Parameters:
-
- config (EPConfig) – The expert-parallel configuration.
- buffers (EPBuffers) – The exchange’s buffers.
- tokens (Tensor) – Each device’s tokens, as for
dispatch(). - ids (Tensor) – Each device’s top-k expert ids, as for
dispatch(). - input_scales (Tensor | None) – As for
dispatch(). - layer_idx (Tensor | None) – As for
dispatch().
-
Returns:
-
The tokens each device received, and the route that
combine_local()sends them back by. -
Raises:
-
ValueError – If
check_supported()rejectsconfig, or ifconfigcounts expert load withoutlayer_idxor withoutEPBuffers.expert_load. -
Return type: