IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

combine_local

combine_local()​

max.nn.ep.combine_local(config, buffers, outputs, route, weights, ids)

source

Returns the experts’ outputs to their tokens, one kernel per device.

With config.fused_shared_expert, the kernel also adds the shared expert’s rows of outputs to each token’s result.

Parameters:

  • config (EPConfig) – The expert-parallel configuration.
  • buffers (EPBuffers) – The exchange’s buffers.
  • outputs (Tensor) – The experts’ outputs, in the rows of the received tokens.
  • route (Route) – The route that dispatch_local() returned.
  • weights (Tensor) – Each device’s router weights, of shape (tokens, top_k) per device, which also give each device’s token count.
  • ids (Tensor) – Each device’s top-k expert ids, which the combine reads with config.use_allreduce.

Returns:

Each token’s weighted sum of its experts’ outputs. With config.use_allreduce, each device holds the sum over its own experts only, so the result is Partial; the tokens must then be the same on every device. Else it is mapped as weights.

Raises:

ValueError – If check_supported() rejects config, or if config.use_allreduce and weights shows that the devices hold different tokens.

Return type:

Tensor