For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.pixtral

Pixtral vision-language architecture for multimodal text generation.

PixtralConfig

class max.pipelines.architectures.pixtral.PixtralConfig(*, dtype, devices, image_token_index, hidden_size, num_attention_heads, rms_norm_eps, rope_theta, max_seq_len, num_hidden_layers, head_dim, num_key_value_heads, feed_forward_length, vocab_size, kv_params, attention_multiplier, patch_size, image_size, num_channels, vision_hidden_size, vision_num_attention_heads, vision_rope_theta, vision_num_hidden_layers, vision_intermediate_size, vision_head_dim, return_logits=ReturnLogits.LAST_TOKEN, quantization_encoding=None)

source

Bases: ArchVLConfigWithTextSubconfig, ArchConfigWithStoredKVParams, ArchConfigWithKVCache

Configuration for Pixtral models.

Parameters:

• dtype (DType)
• devices (list[DeviceRef])
• image_token_index (int)
• hidden_size (int)
• num_attention_heads (int)
• rms_norm_eps (float)
• rope_theta (float)
• max_seq_len (int)
• num_hidden_layers (int)
• head_dim (int)
• num_key_value_heads (int)
• feed_forward_length (int)
• vocab_size (int)
• kv_params (KVCacheParams)
• attention_multiplier (float)
• patch_size (int)
• image_size (int)
• num_channels (int)
• vision_hidden_size (int)
• vision_num_attention_heads (int)
• vision_rope_theta (float)
• vision_num_hidden_layers (int)
• vision_intermediate_size (int)
• vision_head_dim (int)
• return_logits (ReturnLogits)
• quantization_encoding (SupportedEncoding | None)

DEFAULT_ENCODING

DEFAULT_ENCODING: ClassVar[SupportedEncoding] = 'bfloat16'

source

SUPPORTED_ENCODINGS

SUPPORTED_ENCODINGS: ClassVar[set[SupportedEncoding]] = {'bfloat16'}

source

attention_multiplier

attention_multiplier: float

source

calculate_max_seq_len()

classmethod calculate_max_seq_len(pipeline_config, huggingface_config, model_config=None)

source

Delegates to the annotated text config class.

Parameters:

• pipeline_config (PipelineConfig)
• huggingface_config (AutoConfig)
• model_config (MAXModelConfig | None)

Return type:

int

devices

devices: list[DeviceRef]

source

dtype

dtype: DType

source

feed_forward_length

feed_forward_length: int

source

get_num_layers()

static get_num_layers(huggingface_config)

source

Layer count for the decoder stack (override when HF uses a different field).

Parameters:

huggingface_config (AutoConfig)

Return type:

int

head_dim

head_dim: int

source

hidden_size

hidden_size: int

source

image_size

image_size: int

source

image_token_index

image_token_index: int

source

initialize()

classmethod initialize(pipeline_config, model_config=None)

source

Initializes a PixtralConfig instance from pipeline configuration.

This method creates a config instance with all fields that can be determined from the pipeline configuration.

Parameters:

• pipeline_config (PipelineConfig) – The MAX Engine pipeline configuration.
• model_config (MAXModelConfig | None)

Returns:

An initialized PixtralConfig instance.

Return type:

Self

kv_params

kv_params: KVCacheParams

source

max_seq_len

max_seq_len: int

source

num_attention_heads

num_attention_heads: int

source

num_channels

num_channels: int

source

num_hidden_layers

num_hidden_layers: int

source

num_key_value_heads

num_key_value_heads: int

source

patch_size

patch_size: int

source

quantization_encoding

quantization_encoding: SupportedEncoding | None = None

source

return_logits

return_logits: ReturnLogits = 'last_token'

source

Whether to return the last token, all logits, or a variable number of logits.

rms_norm_eps

rms_norm_eps: float

source

rope_theta

rope_theta: float

source

vision_head_dim

vision_head_dim: int

source

vision_hidden_size

vision_hidden_size: int

source

vision_intermediate_size

vision_intermediate_size: int

source

vision_num_attention_heads

vision_num_attention_heads: int

source

vision_num_hidden_layers

vision_num_hidden_layers: int

source

vision_rope_theta

vision_rope_theta: float

source

vocab_size

vocab_size: int

source

PixtralInputs

class max.pipelines.architectures.pixtral.PixtralInputs(tokens, input_row_offsets, return_n_logits, pixel_patches=None, vision_attention_mask=None, vision_position_ids=None, image_token_indices=None, *, kv_cache_inputs=None, lora=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None)

source

Bases: ModelInputs

Holds inputs for the Pixtral model.

Parameters:

• tokens (Buffer)
• input_row_offsets (Buffer)
• return_n_logits (Buffer)
• pixel_patches (Buffer | None)
• vision_attention_mask (Buffer | None)
• vision_position_ids (Buffer | None)
• image_token_indices (Buffer | None)
• kv_cache_inputs (KVCacheInputsInterface[Buffer, Buffer] | None)
• lora (LoRAInputs | None)
• lora_buffers (tuple[Buffer, ...])
• vision_embeddings (list[Buffer])
• vision_scatter_indices (list[Buffer])
• hidden_states (Buffer | list[Buffer] | None)

has_vision_inputs

property has_vision_inputs: bool

source

image_token_indices

image_token_indices: Buffer | None = None

source

input_row_offsets

input_row_offsets: Buffer

source

pixel_patches

pixel_patches: Buffer | None = None

source

return_n_logits

return_n_logits: Buffer

source

tokens

tokens: Buffer

source

vision_attention_mask

vision_attention_mask: Buffer | None = None

source

vision_position_ids

vision_position_ids: Buffer | None = None

source

PixtralModel

class max.pipelines.architectures.pixtral.PixtralModel(pipeline_config, session, devices, kv_cache_config, weights, adapter=None, return_logits=ReturnLogits.LAST_TOKEN, max_batch_size=1)

source

Bases: MultiGraphPipelineModelWithKVCache[TextAndVisionContext]

Pixtral pipeline model with separate vision and language graphs.

Parameters:

• pipeline_config (PipelineConfig)
• session (InferenceSession)
• devices (list[Device])
• kv_cache_config (KVCacheConfig)
• weights (Weights)
• adapter (WeightsAdapter | None)
• return_logits (ReturnLogits)
• max_batch_size (int)

batch_processor_cls

batch_processor_cls

source

alias of PixtralBatchProcessor

calculate_max_seq_len()

classmethod calculate_max_seq_len(pipeline_config, huggingface_config)

source

Bounds max_length by text_config.max_position_embeddings (config is permissive).

Parameters:

• pipeline_config (PipelineConfig)
• huggingface_config (AutoConfig)

Return type:

int

execute()

execute(model_inputs)

source

Executes the graph with the given inputs.

Parameters:

model_inputs (ModelInputs) – The model inputs to execute, containing tensors and any other required data for model execution.

Returns:

ModelOutputs containing the pipeline’s output tensors.

Return type:

ModelOutputs

This is an abstract method that must be implemented by concrete PipelineModels to define their specific execution logic.

language_model

language_model: Model

source

model_config_cls

model_config_cls

source

alias of PixtralConfig

vision_model

vision_model: Model | None

source