For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
EagleProposer
EagleProposer
class max.pipelines.speculative.eagle.EagleProposer(draft, hidden_dim)
Bases: object
Drafts through an EAGLE draft module, one token per step.
The draft is called with the step’s tokens and the hidden state it fuses with their embeddings, and returns logits followed by its own hidden states. Its embedding and head are the target’s.
carry_dim_names
carry_dim_names = CarryDimNames(prefix='', per_device=False)
decode_swaps
decode_swaps: tuple[DecodeKVSwap, ...] = (DecodeKVSwap.MAX_PROMPT_LENGTH_ONE, DecodeKVSwap.DRAFT_ATTENTION_DISPATCH_METADATA)
draft_cache
draft_cache = 'own'
passthrough_decode_swaps
prefill()
prefill(batch, tokens, target_hidden)
Proposes the first draft token from the target’s hidden states.
-
Parameters:
-
- batch (SequentialBatch)
- tokens (TensorValue)
- target_hidden (TensorValue)
-
Return type:
reuse
split_prefix
split_prefix = 'eagle'
step()
step(batch, draft_input, index)
Runs one draft step on the previous step’s token and state.
-
Parameters:
-
- batch (SequentialBatch)
- draft_input (DraftStepInput)
- index (int)
-
Return type:
step_hidden_mode
step_hidden_mode = 'last'
supports_zero_draft_rows
supports_zero_draft_rows = True
uses_thinking_phase
uses_thinking_phase = False