For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
allgather
Multi-GPU allgather implementation that gathers values from multiple GPUs into an output buffer.
This module provides an optimized implementation of allgather operations across multiple GPUs, supporting both peer-to-peer (P2P) and non-P2P communication patterns. The implementation automatically selects between approaches based on hardware capabilities:
-
P2P-based implementation (when P2P access is available):
- Uses direct GPU-to-GPU memory access for better performance.
- Optimized for NVLink and xGMI bandwidth utilization.
- Uses vectorized memory access.
-
Non-P2P fallback implementation:
- Copies data through device memory when direct GPU access isn't possible.
- Simple but functional approach for systems without P2P support.
comptime values
allgather_p2p_tuning_table
comptime allgather_p2p_tuning_table = Table((_allgather_default_configs() + List(DefaultCommTuningConfig(Int(4), Int(8388608), StringSpan("CDNA4"), Int(32)), __list_literal__=NoneType(None))), String("allgather_p2p_table"))
allgather_relay_tuning_table
comptime allgather_relay_tuning_table = Table(List(RelayTuningConfig(Int(-1), Int(-1), Int(72), Int(8), Int(46)), RelayTuningConfig(Int(4), Int(262144), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(262144), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(2147483648), Int(32), Int(4), Int(38)), __list_literal__=NoneType(None)), String("allgather_relay_table"))
allgather_tuning_table
comptime allgather_tuning_table = Table(_allgather_default_configs(), String("allgather_table"))
Functions
-
allgather: Per-device all-gather: one instance per GPU builds its own outputs. -
allgather_rows_slice: Per-device all-gather in which each device keeps only its own row block.