IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

CrossAttentionMTPProposer

CrossAttentionMTPProposer​

class max.pipelines.speculative.cross_attention.CrossAttentionMTPProposer(draft, hidden_dim)

source

Bases: object

Drafts one token a step, attending into the target’s caches.

Parameters:

carry_dim_names​

carry_dim_names = CarryDimNames(prefix='mtp_step', per_device=False)

source

decode_swaps​

decode_swaps: tuple[DecodeKVSwap, ...] = (DecodeKVSwap.MAX_PROMPT_LENGTH_ONE,)

source

draft_cache​

draft_cache = 'target'

source

passthrough_decode_swaps​

passthrough_decode_swaps: tuple[str, ...] = ('full_attention',)

source

prefill()​

prefill(batch, tokens, target_hidden)

source

Runs the first draft step over the target’s hidden states.

Parameters:

Return type:

Proposed

reuse​

reuse: ReuseSpec | None = None

source

split_prefix​

split_prefix = 'mtp'

source

step()​

step(batch, draft_input, index)

source

Runs one later draft step on the previous step’s output.

Parameters:

Return type:

Proposed

step_hidden_mode​

step_hidden_mode = 'last_per_device'

source

supports_zero_draft_rows​

supports_zero_draft_rows = True

source

takes_distributed_inputs​

takes_distributed_inputs = True

source

uses_thinking_phase​

uses_thinking_phase = True

source