IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

DistributedAllGather

struct DistributedAllGather

Registers the mo.distributed.allgather graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[Outs: TypeList[Outs.values], Ins: TypeList[Ins.values], Sigs: TypeList[Sigs.values], //, _trace_name: StaticString, group_size: Int = Int(0), slice_rows: Bool = False](outputs: TensorTuple[*Outs.values], inputs: TensorTuple[*Ins.values], signal_buffers: TensorTuple[*Sigs.values], dev_ctxs_input: DeviceContextArray)

Distributed allgather operation implementation.

Parameters:

  • ​Outs (TypeList[values]): The element types of outputs (inferred).
  • ​Ins (TypeList[values]): The element types of inputs (inferred).
  • ​Sigs (TypeList[values]): The element types of signal_buffers (inferred).
  • ​_trace_name (StaticString): Trace name for profiling.
  • ​group_size (Int): Number of devices per allgather group; must be at least 1 and must evenly divide the total number of devices (defaults to 0).
  • ​slice_rows (Bool): Whether each device receives only its own contiguous block of dim0 / num_devices rows of every gathered piece rather than the whole piece (see allgather_rows_slice). Requires one full-world group.

Args:

Was this page helpful?