For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
p2p_prefill_combine_prep_kernel
def p2p_prefill_combine_prep_kernel[num_threads: Int, src_info_layout: TensorLayout, top_k: Int, n_experts: Int, n_ranks: Int, max_tokens_per_rank: Int, RouterWeightsFnType: RouterWeightsFn, *, SrcInfoEngine: TensorEngine = DefaultEngine](src_info: TileTensor[.int32, src_info_layout, ImmUntrackedOrigin, Engine=SrcInfoEngine], num_tokens: Int32, recv_buf_ptrs: Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks], small_words: MutPointer[UInt32, MutUntrackedOrigin], my_rank: Int32, router_weights_fn: RouterWeightsFnType)
First half of the partial-row combine of a prefill-dispatched step.
Does nothing on a step the P2P small path took. Otherwise it sends this
device's router weights to every device, then its count, and records,
for every (source, token, k) this device computed, the expert-output
row that holds it (from src_info, which the prefill dispatch's place
kernel wrote), tagged with the step's epoch. Nothing is cleared: the
place kernel rewrites the row offsets and marks the padding rows on
every step.
Parameters:
- num_threads (
Int): The number of threads in the block. - src_info_layout (
TensorLayout): The layout of the row sources. - top_k (
Int): Number of experts each token is routed to. - n_experts (
Int): The number of experts over all devices. - n_ranks (
Int): The devices of the node. - max_tokens_per_rank (
Int): The receive buffers' tokens per source device. - RouterWeightsFnType (
RouterWeightsFn): The type ofrouter_weights_fn. - SrcInfoEngine (
TensorEngine): Storage engine backing thesrc_infotile.
Args:
- src_info (
TileTensor[.int32, src_info_layout, ImmUntrackedOrigin, Engine=SrcInfoEngine]):(source token, k)of each expert-output row. - num_tokens (
Int32): The tokens on this device. - recv_buf_ptrs (
Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks]): Every device's combine receive buffer. - small_words (
MutPointer[UInt32, MutUntrackedOrigin]): This device'sEP_P2P_SMALL_WORDS. - my_rank (
Int32): The rank of this device. - router_weights_fn (
RouterWeightsFnType): The router weights of this device's tokens.