IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ep_dispatch_recv_size

def ep_dispatch_recv_size(*, n_experts: Int, n_ranks: Int, max_tokens_per_rank: Int, msg_bytes: Int, token_recv: Bool) -> Int

Returns the bytes of one rank's EP dispatch receive buffer.

Args:

  • ​n_experts (Int): The total number of experts in the model.
  • ​n_ranks (Int): The number of devices participating in communication.
  • ​max_tokens_per_rank (Int): The maximum number of tokens per rank.
  • ​msg_bytes (Int): The token format's message size.
  • ​token_recv (Bool): Whether the dispatch receives each token once per rank (see EPDispatchKernel).

Returns:

Int: The receive buffer's size in bytes.

Was this page helpful?