For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
BundledAllGather
struct BundledAllGather
Registers the mo.bundled.allgather graph op with the graph compiler.
The per-device form of mo.distributed.allgather, full world only: one
instance per GPU gathers every peer's tensor into its own outputs, one
per peer in peer order. execute reads the peers through live data
pointers; build_graph reads them through stable address slots
(Stable inputs).
Implemented traits
Methods
execute
static def execute[Outs: TypeList[Outs.values], Ins: TypeList[Ins.values], Sigs: TypeList[Sigs.values], //, _trace_name: StaticString, peer_rank: Optional[Int] = None](outputs: TensorTuple[*Outs.values], inputs: TensorTuple[*Ins.values], signal_buffers: TensorTuple[*Sigs.values], ctx: DeviceContext)
Per-device allgather through live addresses.
Parameters:
- Outs (
TypeList[values]): The element types ofoutputs(inferred). - Ins (
TypeList[values]): The element types ofinputs(inferred). - Sigs (
TypeList[values]): The element types ofsignal_buffers(inferred). - _trace_name (
StaticString): Trace name used for profiling and debugging. - peer_rank (
Optional[Int]): THIS GPU's index ininputsandsignal_buffers;Nonetakes it from the device id.
Args:
- outputs (
TensorTuple[*Outs.values]): THIS GPU's gathered pieces, one per peer in peer order. - inputs (
TensorTuple[*Ins.values]): Input tensors from ALL participating GPUs. - signal_buffers (
TensorTuple[*Sigs.values]): Signal buffers for ALL participating GPUs. - ctx (
DeviceContext): Device context for THIS GPU.
build_graph
static def build_graph[Outs: TypeList[Outs.values], Ins: TypeList[Ins.values], Sigs: TypeList[Sigs.values], //, _trace_name: StaticString, peer_rank: Optional[Int] = None](outputs: TensorTuple[*Outs.values], inputs: TensorTuple[*Ins.values], signal_buffers: TensorTuple[*Sigs.values], builder: LegacyDeviceGraphBuilder)
Records the per-device allgather into a device graph.
Parameters:
- Outs (
TypeList[values]): The element types ofoutputs(inferred). - Ins (
TypeList[values]): The element types ofinputs(inferred). - Sigs (
TypeList[values]): The element types ofsignal_buffers(inferred). - _trace_name (
StaticString): Trace name used for profiling and debugging. - peer_rank (
Optional[Int]): THIS GPU's index ininputsandsignal_buffers;Nonetakes it from the device id.
Args:
- outputs (
TensorTuple[*Outs.values]): THIS GPU's gathered pieces, one per peer in peer order. - inputs (
TensorTuple[*Ins.values]): Slot-backed input tensors from ALL participating GPUs. - signal_buffers (
TensorTuple[*Sigs.values]): Signal buffers for ALL participating GPUs. - builder (
LegacyDeviceGraphBuilder): Device graph builder for THIS GPU.