IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ep_p2p_small_on

def ep_p2p_small_on[n_experts: Int, top_k: Int, max_token_per_rank: Int, n_gpus_per_node: Int, n_nodes: Int, fused_shared_expert: Bool, skip_a2a: Bool, is_amd: Bool, p2p_small_max_tokens: Int, hidden_size: Int, counter_words: Int]() -> Bool

Returns whether the fused dispatch and combine take the P2P small path.

Both read this one gate, so they always pair. The graph opts in by giving the counter buffer the small path's words, and does so only when every device holds within one token of every other (see ep_fused_dispatch_kernel_api).

Parameters:

  • ​n_experts (Int): Total number of experts.
  • ​top_k (Int): Experts each token is routed to.
  • ​max_token_per_rank (Int): Maximum tokens a device dispatches.
  • ​n_gpus_per_node (Int): Number of GPUs per node.
  • ​n_nodes (Int): Number of physical nodes.
  • ​fused_shared_expert (Bool): Whether the shared expert rides the dispatch.
  • ​skip_a2a (Bool): Whether each device routes within itself.
  • ​is_amd (Bool): Whether the target is an AMD GPU.
  • ​p2p_small_max_tokens (Int): The small path's bound; 0 turns it off.
  • ​hidden_size (Int): Model hidden size, which the dispatch and the combine both know.
  • ​counter_words (Int): Elements of the counter buffer (-1 when dynamic).

Returns:

Bool: Whether the small path's kernels run.

Was this page helpful?