For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
RoutedRowsRMSNormFinalize
struct RoutedRowsRMSNormFinalize
Registers the norm.routed_rows_rms_norm_finalize graph op.
Implemented traits
Methods
execute
static def execute(output: T, input: T, weight: T, expert_offsets: T, restore_order: T, router_weight: T, eps: Float32, ctx: DeviceContext)
Sums each token's normalized routed rows by router weight.
Args:
- output (
T): One row per token[tokens, hidden], float32 or bf16 (rounded once from the float32 sum). - input (
T): The expert outputs in expert order[tokens * top_k, hidden]. - weight (
T): The norm's input scale weight[hidden]. - expert_offsets (
T): Per-group row offsets; the last one is the routed row count. - restore_order (
T): Eachtoken * top_k + kslot's row ininput. - router_weight (
T): Each slot's weight[tokens, top_k]. - eps (
Float32): The norm's epsilon. - ctx (
DeviceContext): Device context.
Raises:
Error: If the launch fails.