For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo package
comm
Provides communication primitives for multi-GPU workloads.
This package implements collective operations: allreduce, allgather, reducescatter, broadcast, and scatter, along with the low-level synchronization signals and Lamport-protocol primitives on which they are built. Both NVIDIA and AMD GPU targets are supported.
These APIs are consumed by the model-serving layer when running distributed inference across multiple GPUs within a node.
Packages
-
vendor: Vendor CCL bindings for multi-GPU collective operations.
Modules
-
allgather: Multi-GPU allgather implementation that gathers values from multiple GPUs into an output buffer. -
allgather_rmsnorm: Fused all-gather + RMSNorm (bf16 in/out, no FP8). -
allreduce: Multi-GPU allreduce implementation for efficient tensor reduction across GPUs. -
allreduce_lamport_rmsnorm: Fused one-shot Lamport allreduce + RMSNorm (high-perf protocol). -
allreduce_residual_rmsnorm: Fused allreduce + RMSNorm (+ optional FP8 quantization) kernel. -
broadcast: Multi-GPU broadcast kernel implementation. -
device_collective: Helpers for dispatching collective operations across devices. -
device_query: Provides device query utilities for communication primitives. -
lamport: Shared primitives for the barrier-free Lamport communication protocol. -
reducescatter: Multi-GPU reducescatter implementation for distributed tensor reduction across GPUs. -
reducescatter_rmsnorm: Fused reduce-scatter + RMSNorm (bf16 in/out, no FP8). -
rms_norm_fp8: Fused RMSNorm + FP8 quantization kernel. -
scatter: Multi-GPU scatter+broadcast kernel implementation. -
sync: Signal-based GPU synchronization primitives for multi-GPU collective kernels.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!