IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

ep_combine_wait_kernel_api

def ep_combine_wait_kernel_api[combine_dtype: DType, hidden_size: Int, top_k: Int, n_experts: Int, max_token_per_rank: Int, n_gpus_per_node: Int, n_nodes: Int, target: StringSpan[ImmStaticOrigin], store_fn: def[dtype: DType, rank: Int, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[rank], SIMD[dtype, width]) capturing thin -> None, router_weights_wrapper: Optional[def[width: Int](token_idx: Int, topk_id: Int) capturing thin -> SIMD[.float32, width]] = None](num_tokens: Int, atomic_counters: TileTensor[.int32, address_space=atomic_counters.address_space, linear_idx_type=atomic_counters.linear_idx_type], recv_ptrs: TileTensor[.uint64, address_space=recv_ptrs.address_space, linear_idx_type=recv_ptrs.linear_idx_type], recv_count_ptrs: TileTensor[.uint64, address_space=recv_count_ptrs.address_space, linear_idx_type=recv_count_ptrs.linear_idx_type], context: DeviceContextBase)

Execute the Expert Parallelism combine completion kernel.

This function launches the combine_wait_kernel from ep_comm.mojo to complete the token combine phase. It waits for all inter-device communication to complete, then computes the weighted sum of routed expert outputs for each token.

Arguments: num_tokens: Number of output tokens. atomic_counters: EP kernel synchronization counters. Used to coordinate between different thread blocks. recv_ptrs: Receive buffer pointers for each local GPU. recv_count_ptrs: Receive count buffer pointers for each local GPU. context: Device context pointer.

Parameters:

  • ​combine_dtype (DType): Data type for tokens during combine phase.
  • ​hidden_size (Int): Model hidden dimension size.
  • ​top_k (Int): Number of experts each token was routed to.
  • ​n_experts (Int): Total experts across all devices.
  • ​max_token_per_rank (Int): Maximum tokens any device can receive.
  • ​n_gpus_per_node (Int): GPUs per physical node.
  • ​n_nodes (Int): Number of physical nodes.
  • ​target (StringSpan[ImmStaticOrigin]): Target.
  • ​store_fn (def[dtype: DType, rank: Int, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[rank], SIMD[dtype, width]) capturing thin -> None): Stores the output tokens. Called with (token_idx, elem) when router_weights_wrapper is set, and with (token_idx, topk_idx, elem) otherwise.
  • ​router_weights_wrapper (Optional[def[width: Int](token_idx: Int, topk_id: Int) capturing thin -> SIMD[.float32, width]]): Wrapper for the optional router weights.

Was this page helpful?