For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

initialize_ep_comm_buffers

initialize_ep_comm_buffers()

max.pipelines.lib.registration_runtime.initialize_ep_comm_buffers(config, runtime, session)

source

Sizes and initializes the expert-parallel communication buffers.

It sizes the buffers from the config, then compiles and runs the expert-parallel init graph, which sets the node ID each device picks its experts by. That is why it runs before the modules are constructed.

Parameters:

• config (ExpertParallelDeploymentConfig) – The finalized config, which sizes the buffers.
• runtime (ModuleRuntime) – The deployment, with ep_size above one and without ep.
• session (InferenceSession) – The session that compiles the init graph.

Returns:

runtime with ModuleRuntime.ep set, and the buffers the decoder’s graph takes, as the config’s expert_parallel_inputs selects them. They are None on virtual devices, which can’t run the init graph.

Raises:

ValueError – If the expert-parallel size isn’t a multiple of the device count.

Return type:

tuple[ModuleRuntime, ExpertParallelExchange | None]