For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
reducescatter
Multi-GPU reducescatter implementation for distributed tensor reduction across GPUs.
comptime values
CollectiveEpilogueFn
comptime CollectiveEpilogueFn = def[dtype: DType, width: SIMDLength, *, alignment: Int](Coord[**?], SIMD[dtype, width]) -> None & RegisterPassable & ImplicitlyCopyable
Collective output epilogue: (coords, value) -> None.
Kernels receive it as a trailing host-layout argument (host_args), so it runs
on the device and must copy ({var}) what it captures.
reducescatter_relay_residual_tuning_table
comptime reducescatter_relay_residual_tuning_table = Table(List(RelayTuningConfig(Int(-1), Int(-1), Int(20), Int(12), Int(36)), RelayTuningConfig(Int(4), Int(131072), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(131072), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(2147483648), Int(20), Int(12), Int(32)), __list_literal__=NoneType(None)), String("reducescatter_relay_residual_table"))
reducescatter_relay_tuning_table
comptime reducescatter_relay_tuning_table = Table(List(RelayTuningConfig(Int(-1), Int(-1), Int(20), Int(12), Int(44)), RelayTuningConfig(Int(4), Int(262144), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(131072), Int(0), Int(0), Int(0)), __list_literal__=NoneType(None)), String("reducescatter_relay_table"))
Structs
-
ReduceScatterConfig: Configuration for axis-aware reduce-scatter partitioning.
Functions
-
reducescatter: Per-device reducescatter operation with axis-aware scatter.