For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
ep_dispatch_recv_size
def ep_dispatch_recv_size(*, n_experts: Int, n_ranks: Int, max_tokens_per_rank: Int, msg_bytes: Int, token_recv: Bool) -> Int
Returns the bytes of one rank's EP dispatch receive buffer.
Args:
- n_experts (
Int): The total number of experts in the model. - n_ranks (
Int): The number of devices participating in communication. - max_tokens_per_rank (
Int): The maximum number of tokens per rank. - msg_bytes (
Int): The token format's message size. - token_recv (
Bool): Whether the dispatch receives each token once per rank (seeEPDispatchKernel).
Returns:
Int: The receive buffer's size in bytes.