IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

moe_finalize_bounded

def moe_finalize_bounded[down_type: DType, weight_type: DType, out_type: DType, //](output: MutTileTensor[out_type, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], down: ImmTileTensor[down_type, Engine=down.Engine, address_space=down.address_space, linear_idx_type=down.linear_idx_type], restore_order: ImmTileTensor[.uint32, Engine=restore_order.Engine, address_space=restore_order.address_space, linear_idx_type=restore_order.linear_idx_type], router_weight: ImmTileTensor[weight_type, Engine=router_weight.Engine, address_space=router_weight.address_space, linear_idx_type=router_weight.linear_idx_type], valid_rows: ImmTileTensor[.uint32, Engine=valid_rows.Engine, address_space=valid_rows.address_space, linear_idx_type=valid_rows.linear_idx_type], context: DeviceContext)

Runs moe_finalize over only the rows of down below valid_rows[0].

A row of down at or past the bound adds nothing to its token's sum and is never read, so it may hold anything, NaN included. Expert-parallel layers sort the slots of experts held on other devices last, past the local rows, and the bound drops them without masking down first.

Parameters:

  • ​down_type (DType): DType of the permuted expert outputs.
  • ​weight_type (DType): DType of the router weights.
  • ​out_type (DType): DType of the combined output.

Args:

Was this page helpful?