For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
routed_rows_rms_norm
An input-scaled RMS norm over the routed rows of an EP expert-output buffer.
out = rms_norm(x * (1 + weight)) # weightless normThe EP dispatch packs a device's routed rows from row 0, and the last
expert offset counts them. That count lives on the device, so the launch
covers the whole static buffer and every row past it returns at once; rows
past it are left unwritten. The body is rms_norm's, on the same launch, so
the routed rows match mo.composite.reduce.rms_norm of the scaled input bit
for bit.
norm.routed_rows_rms_norm_finalize folds that norm into moe.finalize
(nn.routed_rows_finalize): each token sums its routed rows' norms by router
weight, and a slot whose row is past the routed count adds nothing.
Structsโ
- โ
RoutedRowsRMSNorm: Registers thenorm.routed_rows_rms_normgraph op. - โ
RoutedRowsRMSNormFinalize: Registers thenorm.routed_rows_rms_norm_finalizegraph op.