For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo module

distributed

Registers distributed and multi-GPU collective graph ops backed by the comm and shmem kernels.

comptime values

logger

comptime logger = Logger(stdout, prefix=String(""), source_location=False)

Structs

• ​DistributedAllGather: Registers the mo.distributed.allgather graph op with the graph compiler.
• ​DistributedAllGatherRMSNorm: Registers the mo.composite.distributed.allgather_rms_norm graph op with the graph compiler.
• ​DistributedAllReduceAddRMSNormQuantFP8: Registers the mo.composite.distributed.allreduce_add_rms_norm_quant_fp8 graph op with the graph compiler.
• ​DistributedAllReduceSum: Registers the mo.distributed.allreduce.sum graph op with the graph compiler.
• ​DistributedBroadcast: Distributed broadcast: copy tensor from root GPU to all GPUs.
• ​DistributedMatmulReduceScatterSum: Registers the mo.composite.distributed.matmul_reduce_scatter.sum graph op with the graph compiler.
• ​DistributedReduceScatterRMSNorm: Registers the mo.composite.distributed.reduce_scatter_rms_norm graph op with the graph compiler.
• ​DistributedReduceScatterSum: Registers the mo.distributed.reducescatter.sum graph op with the graph compiler.
• ​DistributedScatter: Distributed scatter: send different chunks to different device groups.
• ​LamportAllreduceRMSNorm: Per-rank fused Lamport allreduce + RMSNorm (high-perf protocol).

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit