IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_moe_exit_act_concat

struct Struct_moe_exit_act_concat

Registers the moe.exit.act.concat graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[simd_width: Int](output: T, latent: T, gate_up: T, beta: Float32, linear_beta: Float32, context: DeviceContext)

Writes the normed latent and the shared SiTU-and-mul beside it.

Args:

  • ​output (T): [T, latent_cols + half_cols], the exit GEMM's activation.
  • ​latent (T): [T, latent_cols], the normed latent; a column slice of a wider tensor is read through its view.
  • ​gate_up (T): [T, 2 * half_cols], the shared expert's fused gate/up projection, gate half first.
  • ​beta (Float32): The SiTU gate's tanh scale.
  • ​linear_beta (Float32): The up half's soft saturation, or 0 for none.
  • ​context (DeviceContext): The device context.

Was this page helpful?