For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.mtp_inkling

The MTP depths an Inkling checkpoint ships beside its trunk.

MTPInklingDraftArchConfig

class max.pipelines.architectures.mtp_inkling.MTPInklingDraftArchConfig(*, devices, dtype, kv_params, max_seq_len, text_config, vision_config, quant_config=None, use_subgraphs=True, mtp=None)

source

Bases: InklingConfig

The depths’ config: their target’s, with a KV leaf per attention flavor the depths use.

Parameters:

• devices (list[DeviceRef])
• dtype (DType)
• kv_params (MultiKVCacheParams)
• max_seq_len (int)
• text_config (InklingTextConfig)
• vision_config (InklingVisionConfig)
• quant_config (QuantConfig | None)
• use_subgraphs (bool)
• mtp (InklingMTPConfig | None)

build_module()

classmethod build_module(context)

source

Builds the depths the speculative width needs from the target’s checkpoint, which context names.

They embed and project through the target’s embedding and head, which their proposer shares onto them.

Raises:

ValueError – If the checkpoint ships no MTP depths.

Parameters:

context (ModelLoadContext)

Return type:

tuple[InklingMultiTokenPredictor, MTPInklingDraftArchConfig]

construct_kv_params()

static construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype)

source

Returns the target’s attention leaves, each sized for the depths of its flavor; one depth writes one layer.

Parameters:

• huggingface_config (AutoConfig)
• pipeline_config (PipelineConfig)
• devices (list[DeviceRef])
• kv_cache_config (KVCacheConfig)
• cache_dtype (DType)

Return type:

MultiKVCacheParams

hidden_size

property hidden_size: int

source

Returns the width of the hidden states the depths chain.

convert_inkling_mtp_state_dict()

max.pipelines.architectures.mtp_inkling.convert_inkling_mtp_state_dict(state_dict, huggingface_config, pipeline_config=None, **unused_kwargs)

source

Keeps an Inkling checkpoint’s MTP depths, under the draft’s names.

The depths embed and project through the target’s table and head, so only their own weights load here. Weights for depths past the requested speculative width are dropped without being read.

Parameters:

• state_dict (Mapping[str, Weights])
• huggingface_config (AutoConfig | None)
• pipeline_config (PipelineConfig | None)
• unused_kwargs (object)

Return type:

dict[str, WeightData]