For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.unified_mtp_inkling

Inkling multi-token prediction for unified speculative decoding.

UnifiedMTPInklingInputs

class max.pipelines.architectures.unified_mtp_inkling.UnifiedMTPInklingInputs(tokens, input_row_offsets, positions, return_n_logits, image_embeddings, image_indices, signal_buffers, slot_idx, has_initial_state, conv_pools, *, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, draft_tokens=None, draft_probs_full=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False, sampled_draft_proposal=False, host_input_row_offsets, draft_conv_pools)

source

Bases: UnifiedSpecDecodeInputs, InklingInputs

Inputs for the unified Inkling MTP graph.

Parameters:

• tokens (Buffer)
• input_row_offsets (Buffer)
• positions (Buffer)
• return_n_logits (Buffer)
• image_embeddings (Buffer)
• image_indices (Buffer)
• signal_buffers (list[Buffer])
• slot_idx (list[Buffer])
• has_initial_state (list[Buffer])
• conv_pools (list[Buffer])
• kv_cache_inputs (KVCacheInputsInterface[Buffer, Buffer] | None)
• lora_buffers (tuple[Buffer, ...])
• vision_embeddings (list[Buffer])
• vision_scatter_indices (list[Buffer])
• hidden_states (Buffer | list[Buffer] | None)
• draft_tokens (Buffer | None)
• draft_probs_full (Buffer | None)
• seed (Buffer | None)
• temperature (Buffer | None)
• top_k (Buffer | None)
• max_k (Buffer | None)
• top_p (Buffer | None)
• min_top_p (Buffer | None)
• in_thinking_phase (Buffer | None)
• pinned_bitmask (Buffer | None)
• wait_payload (Buffer | None)
• device_bitmask_scratch (Buffer | None)
• structured_output (bool)
• sampled_draft_proposal (bool)
• host_input_row_offsets (Buffer)
• draft_conv_pools (list[Buffer])

buffers

property buffers: tuple[Buffer, ...]

source

Returns positional Buffer inputs for model ABI calls.

draft_conv_pools

draft_conv_pools: list[Buffer]

source

host_input_row_offsets

host_input_row_offsets: Buffer

source

UnifiedMTPInklingModel

class max.pipelines.architectures.unified_mtp_inkling.UnifiedMTPInklingModel(*args, **kwargs)

source

Bases: _UnifiedSpecDecodeModelMixin, InklingModel

Inkling with MTP: merge + target + rejection + chained draft depths.

batch_processor_cls

batch_processor_cls

source

alias of UnifiedMTPInklingBatchProcessor

release()

release(request_id)

source

Drops the request’s convolution state, freeing its slot.

Parameters:

request_id (RequestID)

Return type:

None