For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
MTPEhNorm
struct MTPEhNorm
Registers the speculative.mtp.eh_norm graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute[hidden_size: Int, block_threads: Int](out_buf: T, embed: T, prev: T, enorm_weight: T, hnorm_weight: T, epsilon: Float32, ctx: DeviceContext)
Normalizes both draft inputs and writes them side by side.
Parameters:
- hidden_size (
Int): Channels per input; the output row is twice this. - block_threads (
Int): Threads per block.
Args:
- out_buf (
T): Output[num_tokens, 2 * hidden_size]. - embed (
T): Token embeddings[num_tokens, hidden_size]. - prev (
T): Target hidden states[num_tokens, hidden_size]. - enorm_weight (
T): Embedding norm weight[hidden_size]. - hnorm_weight (
T): Hidden-state norm weight[hidden_size]. - epsilon (
Float32): Added inside the square root. - ctx (
DeviceContext): Device context.
Raises:
Error: If the target is not a GPU.