For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
combine_wait_kernel
def combine_wait_kernel[output_type: DType, num_threads: Int, n_sms: Int, top_k: Int, n_experts: Int, n_ranks: Int, msg_bytes: Int, max_tokens_per_rank: Int, StoreFnType: EPCombineStoreFn, RouterWeightsFnType: RouterWeightsFn, has_router_weights: Bool, mx_msg: Bool = False](num_tokens: Int32, recv_buf_p: MutPointer[UInt8, MutUntrackedOrigin], recv_count_p: MutPointer[UInt64, MutUntrackedOrigin], ep_counters: EPLocalSyncCounters[n_experts], my_rank: Int32, store_fn: StoreFnType, router_weights_fn: RouterWeightsFnType)
This kernel is called after the combine_kernel to complete the communication. It will keep polling the receive count buffer, and once the count is no longer MAX_FINITE, it can confirm that the communication is complete from a remote rank.
Parameters:
- output_type (
DType): The type of the output tokens. - num_threads (
Int): The number of threads in the block. - n_sms (
Int): The total number of SMs in the device. - top_k (
Int): The number of selected experts per token. - n_experts (
Int): The number of experts in the device. - n_ranks (
Int): The number of ranks. - msg_bytes (
Int): The number of bytes in the message for each token. - max_tokens_per_rank (
Int): The maximum number of tokens per rank. - StoreFnType (
EPCombineStoreFn): The type ofstore_fn. - RouterWeightsFnType (
RouterWeightsFn): The type ofrouter_weights_fn. - has_router_weights (
Bool): Whether all routed experts' outputs for a token are weighted byrouter_weights_fnand summed. - mx_msg (
Bool): Whether each received message is an MXFP8 token.
Args:
- num_tokens (
Int32): The number of output tokens. - recv_buf_p (
MutPointer[UInt8, MutUntrackedOrigin]): The pointer to the receive buffer. Need to be allocated usingshmem_alloc. The underlying buffer is of shape(max_tokens_per_rank, top_k, msg_bytes). - recv_count_p (
MutPointer[UInt64, MutUntrackedOrigin]): The pointer to the receive count buffer. Need to be allocated usingshmem_allocifuse_shmemis True. The underlying buffer is of shape(n_local_experts, n_ranks). - ep_counters (
EPLocalSyncCounters[n_experts]): EP atomic counters for kernel synchronization. - my_rank (
Int32): The rank of the current device. - store_fn (
StoreFnType): Stores the output tokens, passed by host layout. Called with(token_idx, elem)whenhas_router_weights, and with(token_idx, topk_idx, elem)otherwise. - router_weights_fn (
RouterWeightsFnType): The router weights, passed by host layout and called only whenhas_router_weights.