IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

reducescatter

Multi-GPU reducescatter implementation for distributed tensor reduction across GPUs.

comptime values​

CollectiveEpilogueFn​

comptime CollectiveEpilogueFn = def[dtype: DType, width: SIMDLength, *, alignment: Int](Coord[**?], SIMD[dtype, width]) -> None & RegisterPassable & ImplicitlyCopyable

Collective output epilogue: (coords, value) -> None.

Kernels receive it as a trailing host-layout argument (host_args), so it runs on the device and must copy ({var}) what it captures.

reducescatter_relay_residual_tuning_table​

comptime reducescatter_relay_residual_tuning_table = Table(List(RelayTuningConfig(Int(-1), Int(-1), Int(20), Int(12), Int(36)), RelayTuningConfig(Int(4), Int(131072), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(131072), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(2147483648), Int(20), Int(12), Int(32)), __list_literal__=NoneType(None)), String("reducescatter_relay_residual_table"))

reducescatter_relay_tuning_table​

comptime reducescatter_relay_tuning_table = Table(List(RelayTuningConfig(Int(-1), Int(-1), Int(20), Int(12), Int(44)), RelayTuningConfig(Int(4), Int(262144), Int(0), Int(0), Int(0)), RelayTuningConfig(Int(2), Int(131072), Int(0), Int(0), Int(0)), __list_literal__=NoneType(None)), String("reducescatter_relay_table"))

Structs​

Functions​

Was this page helpful?