For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.lfm3

LiquidAI LFM3 models on ModuleV3.

Lfm3

class max.pipelines.architectures.lfm3.Lfm3(config)

source

Bases: Lfm3Base

LFM3 for causal language modeling, as the DECODER role.

Parameters:

config (Lfm3Config)

forward()

forward(inputs)

source

Runs the decoder over the tokens and returns the logits.

Parameters:

inputs (DecoderInputs)

Return type:

ModuleV3Outputs

Lfm3Config

class max.pipelines.architectures.lfm3.Lfm3Config(*, vocab_size, hidden_size, intermediate_size, layer_types, num_attention_heads, num_key_value_heads, head_dim, conv_kernel_size, sliding_window, norm_eps, rope_theta, partial_rotary_factor, use_sandwich_norm, dtype, devices, max_seq_len, kv_params, return_logits=ReturnLogits.LAST_TOKEN)

source

Bases: ArchConfigWithStoredKVParams, ArchConfigWithKVCache

Configuration for an LFM3 decoder.

Each layer is a gated short convolution, sliding-window attention or global attention, chosen by layer_types.

Parameters:

• vocab_size (int)
• hidden_size (int)
• intermediate_size (int)
• layer_types (list[str])
• num_attention_heads (int)
• num_key_value_heads (int)
• head_dim (int)
• conv_kernel_size (int)
• sliding_window (int)
• norm_eps (float)
• rope_theta (dict[str, float])
• partial_rotary_factor (dict[str, float])
• use_sandwich_norm (bool)
• dtype (DType)
• devices (list[DeviceRef])
• max_seq_len (int)
• kv_params (MultiKVCacheParams)
• return_logits (ReturnLogits)

DEFAULT_ENCODING

DEFAULT_ENCODING: ClassVar[max.pipelines.lib.config.SupportedEncoding] = 'bfloat16'

source

SUPPORTED_ENCODINGS

SUPPORTED_ENCODINGS: ClassVar[set[max.pipelines.lib.config.SupportedEncoding]] = {'bfloat16'}

source

calculate_max_seq_len()

classmethod calculate_max_seq_len(huggingface_config, model_config)

source

Returns the max sequence length the text config allows.

Parameters:

• huggingface_config (AutoConfig)
• model_config (MAXModelConfig)

Return type:

int

construct_kv_params()

classmethod construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype, *, allow_kv_head_replication=False)

source

Returns the sliding and full KV caches beside the conv windows.

Each kind of layer indexes its own cache 0, 1, 2, … in layer order. A vision-language config contributes its text half.

Parameters:

• huggingface_config (AutoConfig)
• pipeline_config (PipelineConfig)
• devices (list[DeviceRef])
• kv_cache_config (KVCacheConfig)
• cache_dtype (DType)
• allow_kv_head_replication (bool)

Return type:

MultiKVCacheParams

conv_kernel_size

conv_kernel_size: int

source

devices

devices: list[DeviceRef]

source

dtype

dtype: DType

source

finalize()

finalize(huggingface_config, state_dict, outputs)

source

Returns the config completed with the pipeline’s output requirements.

Parameters:

• huggingface_config (AutoConfig) – The checkpoint’s model configuration.
• state_dict (Mapping[str, WeightData]) – The adapted checkpoint weights.
• outputs (ModelOutputConfig) – The outputs the pipeline requires.

Returns:

A copy of this config that returns the logits outputs asks for.

Return type:

Self

get_num_layers()

static get_num_layers(huggingface_config)

source

Returns the text config’s decoder layer count.

Parameters:

huggingface_config (AutoConfig)

Return type:

int

head_dim

head_dim: int

source

hidden_size

hidden_size: int

source

initialize()

classmethod initialize(pipeline_config, model_config=None, *, max_seq_len)

source

Builds the config from the checkpoint’s text config.

Parameters:

• pipeline_config (PipelineConfig)
• model_config (MAXModelConfig | None)
• max_seq_len (int)

Return type:

Self

intermediate_size

intermediate_size: int

source

kv_params

kv_params: MultiKVCacheParams

source

layer_types

layer_types: list[str]

source

max_seq_len

max_seq_len: int

source

norm_eps

norm_eps: float

source

num_attention_heads

num_attention_heads: int

source

num_key_value_heads

num_key_value_heads: int

source

partial_rotary_factor

partial_rotary_factor: dict[str, float]

source

return_logits

return_logits: ReturnLogits = 'last_token'

source

rope_theta

rope_theta: dict[str, float]

source

sliding_window

sliding_window: int

source

use_sandwich_norm

use_sandwich_norm: bool

source

vocab_size

vocab_size: int

source