IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

p2p_small_combine_kernel

def p2p_small_combine_kernel[input_type: DType, num_threads: Int, input_tokens_layout: TensorLayout, src_info_layout: TensorLayout, top_k: Int, n_ranks: Int, max_tokens_per_rank: Int, StoreFnType: EPCombineStoreFn, RouterWeightsFnType: RouterWeightsFn, has_router_weights: Bool, fused_shared_expert: Bool, small_max_tokens: Int, *, InputTokensEngine: TensorEngine = DefaultEngine, SrcInfoEngine: TensorEngine = DefaultEngine](input_tokens: TileTensor[input_type, input_tokens_layout, ImmUntrackedOrigin, Engine=InputTokensEngine], src_info: TileTensor[.int32, src_info_layout, ImmUntrackedOrigin, Engine=SrcInfoEngine], num_tokens: Int32, recv_buf_ptrs: Array[Pointer[UInt8, MutUntrackedOrigin], n_ranks], small_words: MutPointer[UInt32, MutUntrackedOrigin], my_rank: Int32, store_fn: StoreFnType, router_weights_fn: RouterWeightsFnType)

The combine paired with p2p_small_dispatch_kernel, in one launch.

Does nothing unless that dispatch took the small path. Each routed row goes back to its source device's combine receive buffer at the (token, top-k) slot, followed by a flag tagged with the dispatch's epoch. Then one warp per (token, chunk) of this device's tokens waits for its token's top_k flags and reduces the slots as combine_kernel does, adding the shared expert's row when fused.

Parameters:

  • ​input_type (DType): The type of the expert outputs.
  • ​num_threads (Int): The number of threads in the block.
  • ​input_tokens_layout (TensorLayout): The layout of the expert outputs.
  • ​src_info_layout (TensorLayout): The layout of the row sources.
  • ​top_k (Int): Number of experts each token is routed to.
  • ​n_ranks (Int): The devices of the node.
  • ​max_tokens_per_rank (Int): The combine receive buffer's tokens.
  • ​StoreFnType (EPCombineStoreFn): The type of store_fn.
  • ​RouterWeightsFnType (RouterWeightsFn): The type of router_weights_fn.
  • ​has_router_weights (Bool): Whether the routed outputs are weighted and summed; otherwise each slot's output is stored on its own.
  • ​fused_shared_expert (Bool): Whether to add the shared expert's output.
  • ​small_max_tokens (Int): Must match the dispatch's.
  • ​InputTokensEngine (TensorEngine): Storage engine backing the input_tokens tile.
  • ​SrcInfoEngine (TensorEngine): Storage engine backing the src_info tile.

Args:

Was this page helpful?