IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

p2p_prefill_combine_prep_kernel

def p2p_prefill_combine_prep_kernel[num_threads: Int, src_info_layout: TensorLayout, top_k: Int, n_experts: Int, n_ranks: Int, max_tokens_per_rank: Int, RouterWeightsFnType: RouterWeightsFn, *, SrcInfoEngine: TensorEngine = DefaultEngine](src_info: TileTensor[.int32, src_info_layout, ImmUntrackedOrigin, Engine=SrcInfoEngine], num_tokens: Int32, recv_buf_ptrs: Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks], small_words: MutPointer[UInt32, MutUntrackedOrigin], my_rank: Int32, router_weights_fn: RouterWeightsFnType)

First half of the partial-row combine of a prefill-dispatched step.

Does nothing on a step the P2P small path took. Otherwise it sends this device's router weights to every device, then its count, and records, for every (source, token, k) this device computed, the expert-output row that holds it (from src_info, which the prefill dispatch's place kernel wrote), tagged with the step's epoch. Nothing is cleared: the place kernel rewrites the row offsets and marks the padding rows on every step.

Parameters:

  • ​num_threads (Int): The number of threads in the block.
  • ​src_info_layout (TensorLayout): The layout of the row sources.
  • ​top_k (Int): Number of experts each token is routed to.
  • ​n_experts (Int): The number of experts over all devices.
  • ​n_ranks (Int): The devices of the node.
  • ​max_tokens_per_rank (Int): The receive buffers' tokens per source device.
  • ​RouterWeightsFnType (RouterWeightsFn): The type of router_weights_fn.
  • ​SrcInfoEngine (TensorEngine): Storage engine backing the src_info tile.

Args:

Was this page helpful?