IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

CrossAttentionModuleV3MTPProposer

CrossAttentionModuleV3MTPProposer​

class max.pipelines.speculative.cross_attention.CrossAttentionModuleV3MTPProposer(draft, hidden_dim)

source

Bases: TensorAdapter

Drafts one token a step on ModuleV3, attending into the target’s caches.

The ModuleV3 counterpart of CrossAttentionMTPProposer: the same swaps, carries and outputs, but its draft takes Tensor values and its modes as arguments rather than as attributes it holds.

Parameters:

carry_dim_names​

carry_dim_names = CarryDimNames(prefix='mtp_step', per_device=False)

source

decode_swaps​

decode_swaps: tuple[DecodeKVSwap, ...] = (DecodeKVSwap.MAX_PROMPT_LENGTH_ONE,)

source

draft_cache​

draft_cache = 'target'

source

passthrough_decode_swaps​

passthrough_decode_swaps: tuple[str, ...] = ('full_attention',)

source

prefill()​

prefill(batch, tokens, target_hidden)

source

Runs the first draft step over the target’s hidden states.

Parameters:

Return type:

Proposed[Tensor]

reuse​

reuse: ReuseSpec | None = None

source

split_prefix​

split_prefix = 'mtp'

source

step()​

step(batch, draft_input, index)

source

Runs one later draft step on the previous step’s output.

Parameters:

Return type:

Proposed[Tensor]

step_hidden_mode​

step_hidden_mode = 'last_per_device'

source

supports_zero_draft_rows​

supports_zero_draft_rows = True

source

takes_distributed_inputs​

takes_distributed_inputs = True

source

uses_thinking_phase​

uses_thinking_phase = True

source