For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ExpertParallelDeploymentConfig

ExpertParallelDeploymentConfig

class max.pipelines.lib.registration_runtime.ExpertParallelDeploymentConfig

source

Bases: DeploymentConfig, ABC

A registered config whose decoder serves its experts across devices.

Under expert parallelism the framework sizes the communication buffers with expert_parallel_config(), initializes them, and passes the result to apply_runtime() as ModuleRuntime.ep.

ep_dispatch_token_layout

ep_dispatch_token_layout: ClassVar[str | None] = None

source

How tokens reach the expert dispatch, when the decoder fixes it. Exporters read it off the pipeline model to pick the matching expert-parallel backend.

expert_parallel_config()

abstract expert_parallel_config(runtime)

source

Sizes the expert-parallel buffers from the checkpoint’s experts.

Parameters:

runtime (ModuleRuntime)

Return type:

EPConfig

expert_parallel_inputs

expert_parallel_inputs: ClassVar[ExpertParallelInputs] = <max.pipelines.lib.registration_runtime.ExpertParallelInputs object>

source

How the decoder’s graph takes the exchange’s buffers.