IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

RoutedRowsRMSNormFinalize

struct RoutedRowsRMSNormFinalize

Registers the norm.routed_rows_rms_norm_finalize graph op.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute(output: T, input: T, weight: T, expert_offsets: T, restore_order: T, router_weight: T, eps: Float32, ctx: DeviceContext)

Sums each token's normalized routed rows by router weight.

Args:

  • ​output (T): One row per token [tokens, hidden], float32 or bf16 (rounded once from the float32 sum).
  • ​input (T): The expert outputs in expert order [tokens * top_k, hidden].
  • ​weight (T): The norm's input scale weight [hidden].
  • ​expert_offsets (T): Per-group row offsets; the last one is the routed row count.
  • ​restore_order (T): Each token * top_k + k slot's row in input.
  • ​router_weight (T): Each slot's weight [tokens, top_k].
  • ​eps (Float32): The norm's epsilon.
  • ​ctx (DeviceContext): Device context.

Raises:

Error: If the launch fails.

Was this page helpful?