For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

relay

Shared scaffolding for the relay-assisted grouped collectives.

Two groups of GPUs running a grouped collective concurrently use only their intra-group links; every link between the groups sits idle. Where an even number of groups lets them pair off, a GPU in the partner group can put those links to work as a relay, taking on a trailing fraction of the traffic. The allgather relay is a multicast node and the reduce-scatter relay is a reduce node, but both need the same gate, the same shard-splitting rule and the same shape of tuning entry, which live here.

comptime values

RELAY_ARCH

comptime RELAY_ARCH = StringSpan("CDNA4")

The only architecture the relay-assisted allgather is enabled on.

The transport is generic, but the recipe below was measured on MI355X and the win depends on interconnect specifics -- how much a relay's second hop costs against a direct one -- that do not carry across vendors. _allgather_p2p gates on this, so every tuning row is implicitly for this arch and none of them names it.

Structs

• ​RelayTuningConfig: Tuning-table entry for a relay-assisted grouped collective.