IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

routed_rows_rms_norm

An input-scaled RMS norm over the routed rows of an EP expert-output buffer.

out = rms_norm(x * (1 + weight))          # weightless norm

The EP dispatch packs a device's routed rows from row 0, and the last expert offset counts them. That count lives on the device, so the launch covers the whole static buffer and every row past it returns at once; rows past it are left unwritten. The body is rms_norm's, on the same launch, so the routed rows match mo.composite.reduce.rms_norm of the scaled input bit for bit.

norm.routed_rows_rms_norm_finalize folds that norm into moe.finalize (nn.routed_rows_finalize): each token sums its routed rows' norms by router weight, and a slot whose row is past the routed count adds nothing.

Structsโ€‹

Was this page helpful?