IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_ep_combine_wait_mx

struct Struct_ep_combine_wait_mx

Registers the ep.combine_wait.mx graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[combine_dtype: DType, //, hidden_size: Int, top_k: Int, n_experts: Int, max_token_per_rank: Int, n_gpus_per_node: Int, n_nodes: Int, *, mx_min_tokens: Int](output_tokens: T, atomic_counters: T, recv_ptrs: T, recv_count_ptrs: T, router_weights: T, total_tokens: UInt32, context: DeviceContext)

ep.combine_wait for the messages ep.combine_async.mx sent.

Reads MXFP8 messages when total_tokens reaches mx_min_tokens and exact ones otherwise, then computes each token's weighted sum of its routed expert outputs.

Parameters:

  • ​combine_dtype (DType): Output dtype (inferred).
  • ​hidden_size (Int): Model hidden size.
  • ​top_k (Int): Experts each token is routed to.
  • ​n_experts (Int): Experts across all GPUs.
  • ​max_token_per_rank (Int): Most tokens any GPU can receive.
  • ​n_gpus_per_node (Int): GPUs per physical node.
  • ​n_nodes (Int): Physical nodes.
  • ​mx_min_tokens (Int): Smallest total_tokens sent as MXFP8.

Args:

  • ​output_tokens (T): [num_tokens, hidden_size] weighted sums.
  • ​atomic_counters (T): Atomic counters coordinating the combine.
  • ​recv_ptrs (T): Receive buffer pointers for the combine phase.
  • ​recv_count_ptrs (T): Receive count buffer pointers.
  • ​router_weights (T): [num_tokens, top_k] router weights.
  • ​total_tokens (UInt32): Tokens of the step across all ranks.
  • ​context (DeviceContext): GPU device context for the current device.

Was this page helpful?