For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

GatedDeltaReplayInputs

GatedDeltaReplayInputs

class max.nn.legacy.state_space.GatedDeltaReplayInputs(qkv, conv_weight, decay, beta, conv_output, qkv_window=None)

source

Bases: NamedTuple

One layer’s per-token inputs to the two state kernels.

Speculative decoding runs the verify pass over K+1 positions and then has to land the pools on the accepted prefix instead. Every op feeding these tensors is causal or pointwise, so re-running the two kernels over the accepted rows from the pre-verify state reproduces the state the verify pass held at that length. Captured only when the caller asks for it.

Create new instance of GatedDeltaReplayInputs(qkv, conv_weight, decay, beta, conv_output, qkv_window)

Parameters:

• qkv (TensorValue)
• conv_weight (TensorValue)
• decay (TensorValue)
• beta (TensorValue)
• conv_output (TensorValue)
• qkv_window (TensorValue | None)

beta

beta: TensorValue

source

[total_seq_len, num_value_heads] float32 beta gates.

conv_output

conv_output: TensorValue

source

[total_seq_len, conv_dim] float32 activated conv output.

The conv is causal and a verify reads the window as it was before it ran, so a replay of the recurrence reads its rows here instead of running the conv again.

conv_weight

conv_weight: TensorValue

source

[conv_dim, kernel_size] float32 depthwise weights.

decay

decay: TensorValue

source

[total_seq_len, num_value_heads] float32 decays.

qkv

qkv: TensorValue

source

[total_seq_len, conv_dim] float32 conv input.

qkv_window

qkv_window: TensorValue | None

source

[batch_size * (1 + K), conv_dim] float32 copy of qkv that a verify’s conv kernel writes when there are drafts, or None outside a verify. A conv replay that reads this instead of qkv lets the next layer reuse qkv’s memory.