IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

lamport_allreduce_rmsnorm_fits

def lamport_allreduce_rmsnorm_fits[dtype: DType, ngpus: Int](rows: Int, cols: Int) -> Bool

Returns whether a [rows, cols] message fits the Lamport fused kernel.

The kernel needs whole 128-bit packs, at most one pack per thread of a row's blocks, and a message within the reserved Lamport workspace.

Parameters:

  • ​dtype (DType): Element type of the message.
  • ​ngpus (Int): Number of participating GPUs.

Args:

  • ​rows (Int): Number of rows (tokens).
  • ​cols (Int): Number of columns (hidden size); RMSNorm runs over it.

Returns:

Bool: True when the kernel accepts the shape.

Was this page helpful?