For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
RoutedRowsFinalizeArgs
struct RoutedRowsFinalizeArgs[out_dtype: DType = .float32]
The combine's operands, one kernel parameter. Rows are dense.
Fields
- down (
In[.bfloat16]): Expert outputs[tokens * top_k, HIDDEN], in expert order. - gamma (
In[.bfloat16]): The norm's input scale weight[HIDDEN]. - offsets (
In[.uint32]): Per-group row offsets;offsets[last]is the routed row count. - restore (
In[.uint32]): Eachtoken * top_k + kslot's row indown. - weight (
In[.float32]): Each slot's router weight[tokens, top_k]. - out (
Out[out_dtype]): One combined row per token[tokens, HIDDEN]. - tokens (
Int64): - last (
Int64): - eps (
Float32):
Implemented traits
AnyType,
Copyable,
Deinitable,
DevicePassable,
ImplicitlyCopyable,
Movable
comptime members
device_type
comptime device_type = RoutedRowsFinalizeArgs[out_dtype]