IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo trait

ExpertLoadSink

Receives the routed-token count of each local expert at dispatch wait.

Implementations: NullExpertLoad (the default, counts nothing) and ExpertLoadTally.

Implemented traits​

AnyType, Copyable, Deinitable, DevicePassable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

Required methods​

add​

def add(self, local_expert: Int, n_local_experts: Int, count: UInt32)

Adds one step's routed-token count for a local expert.

Args:

  • ​local_expert (Int): Index of the expert among this rank's experts.
  • ​n_local_experts (Int): Number of routed experts on this rank, the row stride of the tally.
  • ​count (UInt32): Tokens routed to the expert in this step, before any grouped-matmul padding.

Was this page helpful?