For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

UnifiedSpecDecodeInputs

UnifiedSpecDecodeInputs

class max.pipelines.lib.UnifiedSpecDecodeInputs(*, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, draft_tokens=None, draft_probs_full=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, draft_slot_ids=None, draft_block_offsets=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False)

source

Bases: ModelInputs

Shared spec-decode fields + buffer-tail packing for unified *Inputs.

Each arch composes the tail via _spec_decode_tail_buffers(), which mirrors build_spec_decode_input_types and must stay in lockstep with it.

Parameters:

• kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
• lora_buffers (tuple[Buffer, ...])
• vision_embeddings (list[Buffer])
• vision_scatter_indices (list[Buffer])
• hidden_states (Buffer | list[Buffer] | None)
• draft_tokens (Buffer | None)
• draft_probs_full (Buffer | None)
• seed (Buffer | None)
• temperature (Buffer | None)
• top_k (Buffer | None)
• max_k (Buffer | None)
• top_p (Buffer | None)
• min_top_p (Buffer | None)
• in_thinking_phase (Buffer | None)
• draft_slot_ids (Buffer | None)
• draft_block_offsets (Buffer | None)
• pinned_bitmask (Buffer | None)
• wait_payload (Buffer | None)
• device_bitmask_scratch (Buffer | None)
• structured_output (bool)

device_bitmask_scratch

device_bitmask_scratch: Buffer | None = None

source

draft_block_offsets

draft_block_offsets: Buffer | None = None

source

The draft block’s ragged row offsets: [0, K, 2K, ...] to draft, all zeros to skip.

draft_probs_full

draft_probs_full: Buffer | None = None

source

The distribution each draft_tokens entry was sampled from, [batch_size, num_speculative_tokens, vocab_size]. The pipeline sets it exactly when the compiled graph declares it (draft_proposal="sampled"), so the tail packs it whenever it is present.

draft_slot_ids

draft_slot_ids: Buffer | None = None

source

Which drafter rows this step computes, as the buffer’s extent: arange(batch_size * rows_per_seq) to draft, empty to skip. Set by the pipeline from the verify-width schedule, alongside draft_tokens.

draft_tokens

draft_tokens: Buffer | None = None

source

in_thinking_phase

in_thinking_phase: Buffer | None = None

source

max_k

max_k: Buffer | None = None

source

min_top_p

min_top_p: Buffer | None = None

source

pinned_bitmask

pinned_bitmask: Buffer | None = None

source

seed

seed: Buffer | None = None

source

structured_output

structured_output: bool = False

source

Whether this graph was compiled with constrained-decoding bitmask inputs. Mirrors pipeline_config.needs_bitmask_constraints – the same value that gates the bitmask triple in build_spec_decode_input_types – so the buffer tail and the graph signature derive the decision from one place. Set by each capable module’s prepare_initial_token_inputs.

temperature

temperature: Buffer | None = None

source

top_k

top_k: Buffer | None = None

source

top_p

top_p: Buffer | None = None

source

wait_payload

wait_payload: Buffer | None = None

source