IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

allreduce_rmsnorm_exit_concat

def allreduce_rmsnorm_exit_concat[in_dtype: DType, dtype: DType, ngpus: Int, in_layout: TensorLayout, in_origin: Origin[mut=in_origin.mut], //](input_buffers: Array[TileTensor[in_dtype, in_layout, in_origin], ngpus], gamma: TileTensor[Engine=gamma.Engine, address_space=gamma.address_space, linear_idx_type=gamma.linear_idx_type], gate_up: TileTensor[dtype, Engine=gate_up.Engine, address_space=gate_up.address_space, linear_idx_type=gate_up.linear_idx_type], output: TileTensor[dtype, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], epsilon: Float32, beta: Float32, linear_beta: Float32, rank_sigs: Array[Pointer[Signal, MutAnyOrigin], ngpus], ctx: DeviceContext, my_rank: Optional[Int] = None, force_arm: Int = Int(0))

Fused allreduce sum + RMSNorm + exit SiTU-and-mul concat.

Runs the fused kernel at or below the size gate (EXIT_CONCAT_FUSE_THRESHOLD bytes) and the chain it replaced above it; see the module docstring for both arms and why they are bit-identical. A vendor-CCL build and a host without P2P access run the chain too, with the vendor or the plain allreduce reducing in front of it.

Parameters:

  • ​in_dtype (DType): The partials' element type; bfloat16 or float32.
  • ​dtype (DType): The gated projection's and the output's type; bfloat16 only.
  • ​ngpus (Int): Number of cooperating devices.
  • ​in_layout (TensorLayout): Layout of the input partial TileTensors.
  • ​in_origin (Origin[mut=mut]): Origin of the input partial TileTensors.

Args:

Raises:

On any shape, dtype, rank or arm this op does not implement. A world size the size gate has no threshold for takes the chain.

Was this page helpful?