IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.glm5_next

GLM-5.3-Flash architecture: a multimodal MoE decoder with a vision tower, built on the DeepSeek V3.2 model.

Glm5Next​

class max.pipelines.architectures.glm5_next.Glm5Next(config)

source

Bases: Module

The GLM-5.3-Flash language model graph.

Assembles the period-4 hybrid schedule and the four-stream residual. The attention sublayers and the hyper-connection sites are constructed through the seam types, so this class is complete once the four lanes land their modules and needs no edit to accommodate them.

Parameters:

config (Glm5NextConfig)

attach_layers()​

attach_layers(layers)

source

Installs the constructed decoder layers.

Parameters:

layers (LayerList) – One layer per index, num_hidden_layers plus num_nextn_predict_layers when the MTP draft is built.

Raises:

ValueError – If the count does not match the schedule.

Return type:

None

build_layers()​

build_layers()

source

Constructs the decoder stack from the schedule.

One Glm5NextDecoderLayer per index, each holding the attention sublayer its layer_types entry names and the feed-forward sublayer its index implies. The MTP draft layer, when built, is sparse MLA with mhc=False.

The rotary table is built once and shared by every sparse layer. GLM-5.3-Flash has no rotary embedding, but the 576-wide latent geometry carries 64 rotary columns and the fused ops read that width off freqs_cis.shape[1], so a table of that shape has to exist. Its values are irrelevant – every column it rotates is zero – but its leading dimension is max_seq_len, which is 134 MB at 1M context. Once, not eleven times.

Return type:

LayerList

build_mlp()​

build_mlp(layer_idx)

source

Returns the feed-forward sublayer for layer_idx.

Dense at intermediate_size below first_k_dense_replace, MoE at moe_intermediate_size at and above it. Both clamp before the activation; see layers.mlp for why none of MAX’s three existing clamped-SwiGLU paths can be reused.

Parameters:

layer_idx (int)

Return type:

Module

collapse_streams()​

collapse_streams(streams)

source

Collapses the streams back to [T, hidden] by unweighted mean.

Parameters:

streams (TensorValue)

Return type:

TensorValue

expand_streams()​

expand_streams(hidden)

source

Broadcasts [T, hidden] into the [T, hc_mult, hidden] residual.

Parameters:

hidden (TensorValue)

Return type:

TensorValue

input_types()​

input_types(kv_params)

source

Declares the graph’s inputs, in the order execute supplies them.

Five groups, and the order is the contract:

  1. tokens, device row offsets, host row offsets, return_n_logits, data_parallel_splits
  2. the allreduce signal buffers
  3. both KV cache leaves, flattened – "mla" then "indexer"
  4. batch_context_lengths, one per device

The sparse layers’ cache indices are graph constants rather than inputs, matching deepseekV3_2.py:840 – they are fixed by the schedule, so nothing has to supply them per step.

Groups 1-4 are exactly DeepseekV3Inputs.buffers. The recurrent state needs no group of its own: it is a child of the multi-cache, so its pools and row ids are already inside group 3.

Parameters:

kv_params (KVCacheParamInterface)

Return type:

list[TensorType | BufferType]

layer_type()​

layer_type(layer_idx)

source

Returns the attention family of layer_idx.

The MTP draft layer sits at num_hidden_layers and beyond, past the end of layer_types; it is sparse MLA.

Parameters:

layer_idx (int)

Return type:

str

subgraph_layer_groups()​

subgraph_layer_groups()

source

Groups the decoder layers that can share one compiled subgraph.

A layer’s subgraph signature is fixed by two things: which attention family it belongs to, because that decides its per-step bundle, and whether its feed-forward block is dense or MoE. Both come off the schedule rather than off the period, so a sibling that retunes either regroups itself. Nothing else varies – every layer takes and returns the same hc_mult-wide residual, so unlike DeepSeek-V3.2 no layer has to be peeled to keep a group’s arity uniform.

DeepSeek-V3.2 additionally splits its MoE layers on indexer_types, because a "shared" layer declares no indexer weights. GLM-5.3-Flash’s indexer_types is "full" on all 45 layers, so that split would produce one empty group and is not carried here.

Groups of one are dropped: a subgraph with a single call site costs a boundary and saves no elaboration.

Returns:

One list of layer indices per group, or empty when subgraphs are off. The MTP draft layer is never included – it is the one layer without hyper-connections, so its residual is rank 2 where every other layer’s is rank 3.

Return type:

list[list[int]]

uses_hyper_connections()​

uses_hyper_connections(layer_idx)

source

Whether layer_idx has mHC sites rather than a plain residual add.

Layers 0-44 do; the MTP draft layer at 45 does not. hc_attn_fn appears 45 times in the checkpoint index while input_layernorm appears 46 times, which is how this is established.

Parameters:

layer_idx (int)

Return type:

bool

uses_moe()​

uses_moe(layer_idx)

source

Whether layer_idx has an MoE block rather than a dense MLP.

Layers 0-2 are dense at width intermediate_size (12288); 3 onward, the MTP layer included, are MoE at moe_intermediate_size (2048) with one shared expert and eight of 288 routed.

Parameters:

layer_idx (int)

Return type:

bool

Glm5NextConfig​

class max.pipelines.architectures.glm5_next.Glm5NextConfig(*, dtype, kv_params, devices, use_subgraphs=True, data_parallel_degree=1, quantization_encoding=None, vocab_size=129280, hidden_size=7168, intermediate_size=18432, moe_intermediate_size=2048, moe_layer_freq=1, num_hidden_layers=61, num_attention_heads=128, num_key_value_heads=128, n_shared_experts=1, n_routed_experts=256, routed_scaling_factor=2.5, kv_lora_rank=512, q_lora_rank=1536, qk_rope_head_dim=64, v_head_dim=128, qk_nope_head_dim=128, topk_method='greedy', n_group=8, topk_group=4, num_experts_per_tok=8, first_k_dense_replace=3, norm_topk_prob=True, hidden_act='silu', max_position_embeddings=4096, max_seq_len, rms_norm_eps=1e-06, tie_word_embeddings=False, rope_theta=10000.0, rope_scaling=None, rope_interleave=True, scoring_func='sigmoid', attention_bias=False, attention_dropout=0.0, norm_dtype=bfloat16, gate_dtype=None, correction_bias_dtype=None, max_batch_context_length=131072, quant_config=None, attn_quant_config=None, dense_mlp_layers_without_quant=frozenset({}), ep_config=None, graph_mode='auto', return_logits=ReturnLogits.LAST_TOKEN, return_hidden_states=ReturnHiddenStates.NONE, eagle_aux_hidden_state_layer_ids=None, eplb_profile_enabled=False, unpadded_vocab_size=None, index_head_dim=128, index_n_heads=64, index_topk=2048, indexer_types=<factory>, indexer_rope_interleave=False, kv_b_proj_dtype=None, layer_types=<factory>, linear_num_heads=64, linear_head_dim=128, linear_conv_kernel_dim=4, linear_lower_bound=-5.0, state_pool_dtype=float32, mhc=True, hc_mult=4, hc_sinkhorn_iters=20, hc_eps=1e-06, index_kpool=4, index_kpool_compress=True, index_kpool_always_select_tail=True, mla_use_nope=True, mla_latent_pad_to=None, swiglu_limit=10.0, num_nextn_predict_layers=1, index_share_for_mtp_iteration=True, vision_config=None, image_token_id=None, video_token_id=None, image_start_token_id=None, image_end_token_id=None, video_start_token_id=None, video_end_token_id=None, hf_quantization_config=None, quant_scheme=None, declared_dtype=None)

source

Bases: DeepseekV3_2Config

Configuration for GLM-5.3-Flash.

Parameters:

  • dtype (DType)
  • kv_params (KVCacheParamInterface)
  • devices (list[DeviceRef])
  • use_subgraphs (bool)
  • data_parallel_degree (int)
  • quantization_encoding (SupportedEncoding | None)
  • vocab_size (int)
  • hidden_size (int)
  • intermediate_size (int)
  • moe_intermediate_size (int)
  • moe_layer_freq (int)
  • num_hidden_layers (int)
  • num_attention_heads (int)
  • num_key_value_heads (int)
  • n_shared_experts (int)
  • n_routed_experts (int)
  • routed_scaling_factor (float)
  • kv_lora_rank (int)
  • q_lora_rank (int)
  • qk_rope_head_dim (int)
  • v_head_dim (int)
  • qk_nope_head_dim (int)
  • topk_method (str)
  • n_group (int)
  • topk_group (int)
  • num_experts_per_tok (int)
  • first_k_dense_replace (int)
  • norm_topk_prob (bool)
  • hidden_act (str)
  • max_position_embeddings (int)
  • max_seq_len (int)
  • rms_norm_eps (float)
  • tie_word_embeddings (bool)
  • rope_theta (float)
  • rope_scaling (dict[str, Any] | None)
  • rope_interleave (bool)
  • scoring_func (str)
  • attention_bias (bool)
  • attention_dropout (float)
  • norm_dtype (DType)
  • gate_dtype (DType | None)
  • correction_bias_dtype (DType | None)
  • max_batch_context_length (int)
  • quant_config (QuantConfig | None)
  • attn_quant_config (QuantConfig | None)
  • dense_mlp_layers_without_quant (frozenset[int])
  • ep_config (EPConfig | None)
  • graph_mode (str)
  • return_logits (ReturnLogits)
  • return_hidden_states (ReturnHiddenStates)
  • eagle_aux_hidden_state_layer_ids (list[int] | None)
  • eplb_profile_enabled (bool)
  • unpadded_vocab_size (int | None)
  • index_head_dim (int)
  • index_n_heads (int)
  • index_topk (int)
  • indexer_types (list[str])
  • indexer_rope_interleave (bool)
  • kv_b_proj_dtype (DType | None)
  • layer_types (list[str])
  • linear_num_heads (int)
  • linear_head_dim (int)
  • linear_conv_kernel_dim (int)
  • linear_lower_bound (float)
  • state_pool_dtype (DType | None)
  • mhc (bool)
  • hc_mult (int)
  • hc_sinkhorn_iters (int)
  • hc_eps (float)
  • index_kpool (int)
  • index_kpool_compress (bool)
  • index_kpool_always_select_tail (bool)
  • mla_use_nope (bool)
  • mla_latent_pad_to (int | None)
  • swiglu_limit (float)
  • num_nextn_predict_layers (int)
  • index_share_for_mtp_iteration (bool)
  • vision_config (Glm5NextVisionConfig | None)
  • image_token_id (int | None)
  • video_token_id (int | None)
  • image_start_token_id (int | None)
  • image_end_token_id (int | None)
  • video_start_token_id (int | None)
  • video_end_token_id (int | None)
  • hf_quantization_config (dict[str, Any] | None)
  • quant_scheme (Glm5NextQuantScheme | None)
  • declared_dtype (DType | None)

DEFAULT_ENCODING​

DEFAULT_ENCODING: ClassVar[SupportedEncoding] = 'float8_e4m3fn'

source

SUPPORTED_ENCODINGS​

SUPPORTED_ENCODINGS: ClassVar[set[SupportedEncoding]] = {'float8_e4m3fn'}

source

activation_bytes_per_token()​

activation_bytes_per_token()

source

The number of bytes a single live residual tensor occupies per token.

Return type:

int

calculate_max_seq_len()​

classmethod calculate_max_seq_len(huggingface_config, model_config)

source

Bounds against the text config’s max_position_embeddings.

Parameters:

Return type:

int

compute_dtype​

property compute_dtype: DType

source

Dtype of activations and every unquantized weight.

construct_kv_params()​

static construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype)

source

Builds the MLA, indexer and recurrent-state children of the cache.

The graph’s input types are built from these params, so a state derived any later cannot appear among them.

Parameters:

Return type:

KVCacheParamInterface

conv_dim​

property conv_dim: int

source

Width of the concatenated-QKV causal convolution.

declared_dtype​

declared_dtype: DType | None = None

source

The dtype the checkpoint declares for unquantized tensors.

declared_kda_dims()​

static declared_kda_dims(text_config)

source

Returns the KDA dimensions, keyed by the names HF’s config uses.

The state leaves are declared before an instance exists, so the pools cannot read these off Glm5NextConfig. Both readers go through here instead: sizing the pools off one set of names and the layers off another is not a shape error, just a mis-sized pool.

Parameters:

text_config (AutoConfig)

Return type:

_KdaDims

declared_state_dtype()​

static declared_state_dtype(kv_cache_config)

source

Returns state_dtype as far as it is knowable pre-finalize.

The state leaves are declared before a quantization scheme resolves, and cache_dtype is not a stand-in: --kv-cache-format may put the MLA latent in fp8 while the KDA pools stay float32.

Raises:

ValueError – If --state-pool-dtype names a dtype the pools cannot be stored in.

Parameters:

kv_cache_config (KVCacheConfig)

Return type:

DType

get_num_layers()​

static get_num_layers(huggingface_config)

source

Parameters:

huggingface_config (AutoConfig)

Return type:

int

hc_eps​

hc_eps: float = 1e-06

source

hc_mult​

hc_mult: int = 4

source

Number of parallel residual streams.

hc_sinkhorn_iters​

hc_sinkhorn_iters: int = 20

source

Alternating row/column normalisation steps that project the stream-mix matrix towards doubly stochastic.

hf_quantization_config​

hf_quantization_config: dict[str, Any] | None = None

source

image_end_token_id​

image_end_token_id: int | None = None

source

image_start_token_id​

image_start_token_id: int | None = None

source

image_token_id​

image_token_id: int | None = None

source

index_kpool​

index_kpool: int = 4

source

Tokens per indexer scoring pool.

index_kpool_always_select_tail​

index_kpool_always_select_tail: bool = True

source

Whether the incomplete trailing pool is always visible.

index_kpool_compress​

index_kpool_compress: bool = True

source

Whether pooled keys are a learned weighted average over each pool’s tokens (gate logits plus a learned intra-pool position embedding) rather than a plain mean.

index_share_for_mtp_iteration​

index_share_for_mtp_iteration: bool = True

source

Whether the MTP draft layer reuses the target’s top-k selection.

infer_optimal_batch_size()​

infer_optimal_batch_size(devices, *, weights_size, device_memory_utilization)

source

Returns a memory-safe default max_batch_size.

The state occupies pages of the KV pool, so it is reserved nowhere here. The states get up to half the post-weights budget and the KV absorbs the rest; the halving bounds concurrency, it allocates nothing.

Parameters:

Return type:

int

initialize()​

classmethod initialize(pipeline_config, model_config=None, *, max_seq_len)

source

Builds the config from the checkpoint’s nested text_config.

Parameters:

Return type:

Self

kda_layers​

property kda_layers: tuple[int, ...]

source

Indices of the KDA linear-attention layers.

layer_types​

layer_types: list[str]

source

Per-layer attention family: LINEAR_ATTENTION or SPARSE_ATTENTION.

linear_conv_kernel_dim​

linear_conv_kernel_dim: int = 4

source

Depthwise causal conv1d kernel size over the concatenated QKV.

linear_head_dim​

linear_head_dim: int = 128

source

KDA head dimension, shared by q, k and v. The recurrent state is [linear_num_heads, linear_head_dim, linear_head_dim].

linear_lower_bound​

linear_lower_bound: float = -5.0

source

Lower bound of the bounded forget gate.

linear_num_heads​

linear_num_heads: int = 64

source

KDA heads. One count for q, k and v alike, instead of separate key and value head counts.

mhc​

mhc: bool = True

source

Whether the decoder uses manifold-constrained hyper-connections. False for the MTP layer, which has a plain residual add.

mla_head_dim​

property mla_head_dim: int

source

Latent width the MLA KV cache stores per token.

mla_latent_pad_to​

mla_latent_pad_to: int | None = None

source

Temporarily required padding for MLA kernel.

mla_use_nope​

mla_use_nope: bool = True

source

NoPE. There is no rotary embedding anywhere in the text decoder.

num_nextn_predict_layers​

num_nextn_predict_layers: int = 1

source

MTP draft layers, at indices num_hidden_layers onward.

per_request_state_bytes()​

per_request_state_bytes()

source

Returns GPU bytes one request’s recurrent state occupies.

Summed over every device, and read off the leaves the cache declares, so the budget and the pool cannot disagree about a shape.

Per KDA layer, at state_dtype:

  • recurrent state [linear_num_heads, linear_head_dim, linear_head_dim]
  • conv state [conv_dim, linear_conv_kernel_dim - 1]

At float32 and the real dimensions that is 4 MiB + 288 KiB per layer, or 146 MiB per sequence across 34 layers, independent of context length. The indexer tail rings add ~0.01% on top, once per device rather than split across them.

Return type:

int

quant_scheme​

quant_scheme: Glm5NextQuantScheme | None = None

source

resolve_layer_types()​

static resolve_layer_types(text_config)

source

Returns the per-layer attention family list.

Prefers the checkpoint’s explicit layer_types. Falls back to linear_attn_config.full_attn_layers, then to the period-4 rule the family uses (sparse where idx % 4 == 3).

Parameters:

text_config (AutoConfig)

Return type:

list[str]

resolve_quant_scheme()​

resolve_quant_scheme(state_dict)

source

Builds the mixed-precision map from the checkpoint’s weight scales.

Parameters:

state_dict (Mapping[str, WeightData])

Return type:

QuantConfig | None

sparse_attention_layers​

property sparse_attention_layers: tuple[int, ...]

source

Indices of the sparse-MLA layers, excluding the MTP draft layer.

state_dtype​

property state_dtype: DType

source

Storage dtype of the KDA conv and recurrent pools.

state_pool_dtype​

state_pool_dtype: DType | None = 81

source

Storage dtype of the KDA conv and recurrent pools.

swiglu_limit​

swiglu_limit: float = 10.0

source

Clamp applied before the activation: silu(min(gate, limit)) * clamp(up, -limit, limit).

video_end_token_id​

video_end_token_id: int | None = None

source

video_start_token_id​

video_start_token_id: int | None = None

source

video_token_id​

video_token_id: int | None = None

source

vision_config​

vision_config: Glm5NextVisionConfig | None = None

source

Vision tower configuration; None for a text-only checkpoint.

Glm5NextMemoryPlanner​

class max.pipelines.architectures.glm5_next.Glm5NextMemoryPlanner(config)

source

Bases: DeepseekV3MemoryPlanner

Adds the KDA state pools and the widened mHC residual to V3’s estimate.

Initializes the paged memory planner.

Parameters:

config (Any) – Must implement ModelConfigWithKVCache.

Raises:

TypeError – If config does not satisfy ModelConfigWithKVCache.

describe()​

describe()

source

Returns the per-sequence costs, for logging a capacity decision.

Return type:

dict[str, Any]

estimate_activation_memory()​

estimate_activation_memory(pipeline_config, huggingface_config)

source

Adds the widened residual to V3’s estimate, and nothing else.

The conv and recurrent pools are leaves of the multi-cache, so the cache allocation already pays for their pages – per_request_state_bytes() reads the same leaves the pool is built from. Reserving them here as well would subtract one request’s state twice and shrink the cache by that much, which is why the sibling Qwen3.5 planner reserves nothing for it either.

What is left is the mHC residual, which is genuinely an activation: hc_mult copies of the hidden state, live for the whole stack and owned by no cache. It is replicated, so it is sized from the whole batch. The EP dispatch buffers are not: the feed-forward sublayer splits the token axis, which is what makes the inherited per-rank estimate right without an override here.

Parameters:

Return type:

int

infer_max_batch_size()​

infer_max_batch_size(pipeline_config, devices, weights_size)

source

Infers a memory-safe default max_batch_size.

The framework’s default inference assumes per-request GPU cost is the KV cache alone, which understates GLM-5.3-Flash by 146 MiB per sequence and OOMs on short-context traffic.

Parameters:

Return type:

int | None

Glm5NextModel​

class max.pipelines.architectures.glm5_next.Glm5NextModel(pipeline_config, session, devices, kv_cache_config, weights, *, memory_plan, adapter=None, return_logits=ReturnLogits.ALL, return_hidden_states=ReturnHiddenStates.NONE, max_batch_size=1)

source

Bases: DeepseekV3_2Model

GLM-5.3-Flash pipeline model.

Parameters:

load_model()​

load_model(session)

source

Compiles the model.

The graph is GLM-5.3-Flash’s own; see _build_graph_for_compile().

Parameters:

session (InferenceSession)

Return type:

Any

model_config_cls​

model_config_cls

source

alias of Glm5NextConfig

Glm5NextQuantScheme​

class max.pipelines.architectures.glm5_next.Glm5NextQuantScheme(config, mlp_layers, attn_layers, compute_dtype)

source

Bases: object

Which GLM-5.3-Flash modules are FP8, and how.

Parameters:

attn_config()​

attn_config(layer_idx)

source

Returns the config for layer layer_idx’s attention projections.

Applies to the four FP8 projections only; callers must keep UNQUANTIZED_ATTN_PROJECTIONS at compute_dtype even in a layer this returns a config for.

Parameters:

layer_idx (int)

Return type:

QuantConfig | None

attn_layers​

attn_layers: frozenset[int]

source

Layers whose q_a_proj / q_b_proj / kv_a_proj_with_mqa / o_proj are FP8. On GLM-5.3-Flash this is exactly the sparse-MLA layer subset plus the MTP layer; the 34 KDA layers hold no FP8 weight at all.

compute_dtype​

compute_dtype: DType

source

Dtype of everything unquantized: activations, norms, embeddings, every KDA projection and conv, kv_b_proj, the indexer, the vision tower and the mHC mapping. Distinct from Glm5NextConfig.dtype, which is the storage dtype the resolved encoding implies.

config​

config: QuantConfig

source

Blockwise-FP8 config for the modules the layer sets below name.

mlp_config()​

mlp_config(layer_idx)

source

Returns the config for layer layer_idx’s MLP, or None if BF16.

Parameters:

layer_idx (int)

Return type:

QuantConfig | None

mlp_layers​

mlp_layers: frozenset[int]

source

Layers whose MLP – dense, shared expert and routed experts – is FP8.

Glm5NextVisionConfig​

class max.pipelines.architectures.glm5_next.Glm5NextVisionConfig(*, depth=24, hidden_size=1024, num_heads=16, intermediate_size=4096, projection_intermediate_size=10240, out_hidden_size=4096, patch_size=14, temporal_patch_size=2, spatial_merge_size=2, in_channels=3, image_size=448, rms_norm_eps=1e-05, hidden_act='silu', attention_bias=True, swiglu_limit=10.0, dtype=bfloat16)

source

Bases: object

Vision tower configuration for GLM-5.3-Flash.

Parameters:

  • depth (int)
  • hidden_size (int)
  • num_heads (int)
  • intermediate_size (int)
  • projection_intermediate_size (int)
  • out_hidden_size (int)
  • patch_size (int)
  • temporal_patch_size (int)
  • spatial_merge_size (int)
  • in_channels (int)
  • image_size (int)
  • rms_norm_eps (float)
  • hidden_act (str)
  • attention_bias (bool)
  • swiglu_limit (float)
  • dtype (DType)

attention_bias​

attention_bias: bool = True

source

Every projection in the tower carries a bias, unlike MAX’s GLM-4.6V tower, which sets has_bias=False on both qkv and proj.

depth​

depth: int = 24

source

Number of transformer blocks.

dtype​

dtype: DType = 80

source

The vision tower is BF16 throughout; it carries no FP8 weight.

head_dim​

property head_dim: int

source

hidden_act​

hidden_act: str = 'silu'

source

hidden_size​

hidden_size: int = 1024

source

Block width.

image_size​

image_size: int = 448

source

in_channels​

in_channels: int = 3

source

initialize_from_config()​

classmethod initialize_from_config(hf_vision_config, *, dtype)

source

Reads the tower’s fields off the checkpoint’s vision_config.

Parameters:

  • hf_vision_config (AutoConfig)
  • dtype (DType)

Return type:

Self

intermediate_size​

intermediate_size: int = 4096

source

Width of each block’s MLP. Not the merger’s.

num_heads​

num_heads: int = 16

source

Attention heads per block; head_dim is hidden_size // num_heads.

out_hidden_size​

out_hidden_size: int = 4096

source

Tower output width. Equal to the decoder’s hidden_size, so the merger feeds the decoder directly with no bridge projector.

patch_size​

patch_size: int = 14

source

Spatial patch edge.

projection_intermediate_size​

projection_intermediate_size: int = 10240

source

Width of the patch merger’s clamped SwiGLU.

rms_norm_eps​

rms_norm_eps: float = 1e-05

source

spatial_merge_size​

spatial_merge_size: int = 2

source

Edge of the strided-conv spatial merge.

swiglu_limit​

swiglu_limit: float = 10.0

source

Clamp applied to the block MLP and merger gate/up projections.

temporal_patch_size​

temporal_patch_size: int = 2

source

Temporal patch depth of the 3D patch embedding.

convert_glm5_next_state_dict()​

max.pipelines.architectures.glm5_next.convert_glm5_next_state_dict(state_dict, huggingface_config, **unused_kwargs)

source

Converts a GLM-5.3-Flash checkpoint to MAX weight names.

Parameters:

  • state_dict (dict[str, Weights]) – The raw checkpoint weights.
  • huggingface_config (AutoConfig) – The checkpoint’s HuggingFace config, read for the layer count that identifies the MTP draft layer.
  • unused_kwargs (object)

Returns:

The transformed weights.

Raises:

ValueError – If a KDA layer is missing one of its three conv tensors, or if the three disagree on dtype or kernel width.

Return type:

dict[str, WeightData]

Models

Cookbook

Puzzles

Blog

Community