IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

p2p_prefill_dispatch_count_kernel

def p2p_prefill_dispatch_count_kernel[num_threads: Int, n_experts: Int, n_ranks: Int, max_tokens_per_rank: Int, msg_bytes: Int, top_k: Int, topk_off: Int](recv_buf_ptrs: Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks], small_words: MutPointer[UInt32, MutUntrackedOrigin], my_rank: Int32)

Second launch of the P2P prefill dispatch: waits for every source's token flags and counts this device's rows per local expert.

Parameters:

  • ​num_threads (Int): The number of threads in the block.
  • ​n_experts (Int): The number of experts over all devices.
  • ​n_ranks (Int): The devices of the node.
  • ​max_tokens_per_rank (Int): The receive buffer's tokens per source device.
  • ​msg_bytes (Int): Bytes of one message.
  • ​top_k (Int): Number of experts each token is routed to.
  • ​topk_off (Int): Offset of the top-k ids in a message.

Args:

Was this page helpful?