IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

combine_wait_kernel

def combine_wait_kernel[output_type: DType, num_threads: Int, n_sms: Int, top_k: Int, n_experts: Int, n_ranks: Int, msg_bytes: Int, max_tokens_per_rank: Int, StoreFnType: EPCombineStoreFn, RouterWeightsFnType: RouterWeightsFn, has_router_weights: Bool, mx_msg: Bool = False](num_tokens: Int32, recv_buf_p: MutPointer[UInt8, MutUntrackedOrigin], recv_count_p: MutPointer[UInt64, MutUntrackedOrigin], ep_counters: EPLocalSyncCounters[n_experts], my_rank: Int32, store_fn: StoreFnType, router_weights_fn: RouterWeightsFnType)

This kernel is called after the combine_kernel to complete the communication. It will keep polling the receive count buffer, and once the count is no longer MAX_FINITE, it can confirm that the communication is complete from a remote rank.

Parameters:

  • ​output_type (DType): The type of the output tokens.
  • ​num_threads (Int): The number of threads in the block.
  • ​n_sms (Int): The total number of SMs in the device.
  • ​top_k (Int): The number of selected experts per token.
  • ​n_experts (Int): The number of experts in the device.
  • ​n_ranks (Int): The number of ranks.
  • ​msg_bytes (Int): The number of bytes in the message for each token.
  • ​max_tokens_per_rank (Int): The maximum number of tokens per rank.
  • ​StoreFnType (EPCombineStoreFn): The type of store_fn.
  • ​RouterWeightsFnType (RouterWeightsFn): The type of router_weights_fn.
  • ​has_router_weights (Bool): Whether all routed experts' outputs for a token are weighted by router_weights_fn and summed.
  • ​mx_msg (Bool): Whether each received message is an MXFP8 token.

Args:

  • ​num_tokens (Int32): The number of output tokens.
  • ​recv_buf_p (MutPointer[UInt8, MutUntrackedOrigin]): The pointer to the receive buffer. Need to be allocated using shmem_alloc. The underlying buffer is of shape (max_tokens_per_rank, top_k, msg_bytes).
  • ​recv_count_p (MutPointer[UInt64, MutUntrackedOrigin]): The pointer to the receive count buffer. Need to be allocated using shmem_alloc if use_shmem is True. The underlying buffer is of shape (n_local_experts, n_ranks).
  • ​ep_counters (EPLocalSyncCounters[n_experts]): EP atomic counters for kernel synchronization.
  • ​my_rank (Int32): The rank of the current device.
  • ​store_fn (StoreFnType): Stores the output tokens, passed by host layout. Called with (token_idx, elem) when has_router_weights, and with (token_idx, topk_idx, elem) otherwise.
  • ​router_weights_fn (RouterWeightsFnType): The router weights, passed by host layout and called only when has_router_weights.

Was this page helpful?