For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
p2p_prefill_dispatch_count_kernel
def p2p_prefill_dispatch_count_kernel[num_threads: Int, n_experts: Int, n_ranks: Int, max_tokens_per_rank: Int, msg_bytes: Int, top_k: Int, topk_off: Int](recv_buf_ptrs: Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks], small_words: MutPointer[UInt32, MutUntrackedOrigin], my_rank: Int32)
Second launch of the P2P prefill dispatch: waits for every source's token flags and counts this device's rows per local expert.
Parameters:
- num_threads (
Int): The number of threads in the block. - n_experts (
Int): The number of experts over all devices. - n_ranks (
Int): The devices of the node. - max_tokens_per_rank (
Int): The receive buffer's tokens per source device. - msg_bytes (
Int): Bytes of one message. - top_k (
Int): Number of experts each token is routed to. - topk_off (
Int): Offset of the top-k ids in a message.
Args:
- recv_buf_ptrs (
Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks]): Every device's dispatch receive buffer. - small_words (
MutPointer[UInt32, MutUntrackedOrigin]): This device'sEP_P2P_SMALL_WORDS. - my_rank (
Int32): The rank of this device.