For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.qwen3_embedding

Qwen3 architecture for embeddings generation.

Qwen3EmbeddingConfig

class max.pipelines.architectures.qwen3_embedding.Qwen3EmbeddingConfig(*, pipeline_config, max_seq_len, quantization_encoding=None)

source

Bases: ArchConfig

Qwen3 embedding model configuration.

Parameters:

• pipeline_config (PipelineConfig)
• max_seq_len (int)
• quantization_encoding (Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'float6_e2m3fn', 'gptq'] | None)

DEFAULT_ENCODING

DEFAULT_ENCODING: ClassVar[Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'float6_e2m3fn', 'gptq']] = 'bfloat16'

source

SUPPORTED_ENCODINGS

SUPPORTED_ENCODINGS: ClassVar[set[Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'float6_e2m3fn', 'gptq']]] = {'bfloat16', 'float32'}

source

calculate_max_seq_len()

classmethod calculate_max_seq_len(huggingface_config, model_config)

source

Returns the resolved maximum sequence length.

Bounds or defaults the user’s model_config.max_length with the model’s own limits. Construction runs this once and stores the result on model_config.max_length; memory planning may lower it further, but only on the memory plan.

Parameters:

• huggingface_config (AutoConfig) – The HuggingFace config to read model bounds from.
• model_config (MAXModelConfig) – The model config whose max_length carries the user’s setting.

Return type:

int

get_max_seq_len()

get_max_seq_len()

source

Returns the effective maximum sequence length for the model.

For configs that store a deployment length, this is the value initialize received; for metadata-only configs it derives from the checkpoint.

Return type:

int

initialize()

classmethod initialize(pipeline_config, model_config=None, *, max_seq_len)

source

Initialize the config from a PipelineConfig.

Parameters:

• pipeline_config (PipelineConfig) – The pipeline configuration.
• model_config (MAXModelConfig | None) – The model configuration to read from. When None (the default), pipeline_config.model is used. Pass an explicit config (e.g. pipeline_config.draft_model) to initialize the arch config for a different model.
• max_seq_len (int) – The effective maximum sequence length to store on the config. The value is received, never derived here: the pipeline model passes the memory plan’s VRAM-clamped length, while memory planning (which runs before a plan exists) passes the construction-resolved model_config.max_length. Configs whose sequence length is pure model metadata (e.g. diffusion components) ignore it.

Return type:

Self

max_seq_len

max_seq_len: int

source

pipeline_config

pipeline_config: PipelineConfig

source

quantization_encoding

quantization_encoding: Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'float6_e2m3fn', 'gptq'] | None = None

source

Qwen3EmbeddingInputs

class max.pipelines.architectures.qwen3_embedding.Qwen3EmbeddingInputs(tokens, input_row_offsets, return_n_logits, *, kv_cache_inputs=None, lora_buffers=(), vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None)

source

Bases: ModelInputs

Input structure for Qwen3 embedding models.

Parameters:

• tokens (Buffer)
• input_row_offsets (Buffer)
• return_n_logits (Buffer)
• kv_cache_inputs (KVCacheInputsInterface[Buffer, Buffer] | None)
• lora_buffers (tuple[Buffer, ...])
• vision_embeddings (list[Buffer])
• vision_scatter_indices (list[Buffer])
• hidden_states (Buffer | list[Buffer] | None)

input_row_offsets

input_row_offsets: Buffer

source

Row offsets for ragged tensors [batch_size + 1]

return_n_logits

return_n_logits: Buffer

source

Number of logits to return (kept for interface compatibility)

tokens

tokens: Buffer

source

Input token IDs [total_seq_len]

Qwen3EmbeddingModel

class max.pipelines.architectures.qwen3_embedding.Qwen3EmbeddingModel(pipeline_config, session, devices, kv_cache_config, weights, *, memory_plan, adapter=None, return_logits=ReturnLogits.ALL, max_batch_size=1)

source

Bases: GraphPipelineModel[TextContext]

Qwen3 embedding pipeline model without KV caching.

This model is optimized for embedding generation with:

• No KV cache overhead
• Single-pass forward computation
• Flash attention without cache operations
• Last token pooling with L2 normalization

Initialize the Qwen3 embedding pipeline model.

Parameters:

• pipeline_config (PipelineConfig) – Pipeline configuration
• session (InferenceSession) – Inference session
• devices (list[Device]) – List of devices
• kv_cache_config (KVCacheConfig) – KV cache configuration
• weights (Weights) – Model weights
• adapter (WeightsAdapter | None) – Optional weight adapter
• return_logits (ReturnLogits) – Return logits mode
• memory_plan (MemoryPlan)
• max_batch_size (int)

attention_bias

attention_bias: bool = False

source

Whether to use attention bias.

batch_processor_cls

batch_processor_cls

source

alias of Qwen3EmbeddingBatchProcessor

execute()

execute(model_inputs)

source

Execute the model.

Parameters:

model_inputs (ModelInputs) – Model inputs

Returns:

Model outputs with embeddings in the logits field

Return type:

ModelOutputs

model

model: Model

source

Compiled and initialized model.

model_config_cls

model_config_cls

source

alias of Qwen3EmbeddingConfig

norm_method

norm_method: Literal['rms_norm', 'layer_norm'] = 'rms_norm'

source

Normalization method.

state_dict

state_dict: dict[str, Any]

source

Model weights.