For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

allgather

Multi-GPU allgather implementation that gathers values from multiple GPUs into an output buffer.

This module provides an optimized implementation of allgather operations across multiple GPUs, supporting both peer-to-peer (P2P) and non-P2P communication patterns. The implementation automatically selects between approaches based on hardware capabilities:

1. P2P-based implementation (when P2P access is available):

   • Uses direct GPU-to-GPU memory access for better performance.
   • Optimized for NVLink and xGMI bandwidth utilization.
   • Uses vectorized memory access.

2. Non-P2P fallback implementation:

   • Copies data through device memory when direct GPU access isn't possible.
   • Simple but functional approach for systems without P2P support.

comptime values

allgather_relay_tuning_table

comptime allgather_relay_tuning_table = Table(List(RelayTuningConfig(Int(-1), Int(-1), Int(72), Int(8), Int(46)), RelayTuningConfig(Int(4), Int(262144), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(262144), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(2147483648), Int(32), Int(4), Int(38)), __list_literal__=NoneType(None)), String("allgather_relay_table"))

allgather_tuning_table

comptime allgather_tuning_table = Table(List(DefaultCommTuningConfig(Int(-1), Int(-1), StringSpan("sm_90a"), Int(216)), DefaultCommTuningConfig(Int(-1), Int(-1), StringSpan("sm_100a"), Int(512)), DefaultCommTuningConfig(Int(-1), Int(-1), StringSpan("sm_103a"), Int(512)), DefaultCommTuningConfig(Int(-1), Int(-1), StringSpan("CDNA4"), Int(128)), DefaultCommTuningConfig(Int(-1), Int(-1), StringSpan("default"), Int(512)), __list_literal__=NoneType(None)), String("allgather_table"))

Functions

• ​allgather: Per-device all-gather: one instance per GPU builds its own outputs.