IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MTPEhNorm

struct MTPEhNorm

Registers the speculative.mtp.eh_norm graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[hidden_size: Int, block_threads: Int](out_buf: T, embed: T, prev: T, enorm_weight: T, hnorm_weight: T, epsilon: Float32, ctx: DeviceContext)

Normalizes both draft inputs and writes them side by side.

Parameters:

  • ​hidden_size (Int): Channels per input; the output row is twice this.
  • ​block_threads (Int): Threads per block.

Args:

  • ​out_buf (T): Output [num_tokens, 2 * hidden_size].
  • ​embed (T): Token embeddings [num_tokens, hidden_size].
  • ​prev (T): Target hidden states [num_tokens, hidden_size].
  • ​enorm_weight (T): Embedding norm weight [hidden_size].
  • ​hnorm_weight (T): Hidden-state norm weight [hidden_size].
  • ​epsilon (Float32): Added inside the square root.
  • ​ctx (DeviceContext): Device context.

Raises:

Error: If the target is not a GPU.

Was this page helpful?