For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python module

max.pipelines.architectures.glm5_1

GLM-5.1 (GlmMoeDsa) mixture-of-experts architecture for text generation.

Glm5_1Config

class max.pipelines.architectures.glm5_1.Glm5_1Config(*, dtype, kv_params, devices, use_subgraphs=True, data_parallel_degree=1, quantization_encoding=None, vocab_size=129280, hidden_size=7168, intermediate_size=18432, moe_intermediate_size=2048, moe_layer_freq=1, num_hidden_layers=61, num_attention_heads=128, num_key_value_heads=128, n_shared_experts=1, n_routed_experts=256, routed_scaling_factor=2.5, kv_lora_rank=512, q_lora_rank=1536, qk_rope_head_dim=64, v_head_dim=128, qk_nope_head_dim=128, topk_method='greedy', n_group=8, topk_group=4, num_experts_per_tok=8, first_k_dense_replace=3, norm_topk_prob=True, hidden_act='silu', max_position_embeddings=4096, max_seq_len, rms_norm_eps=1e-06, tie_word_embeddings=False, rope_theta=10000.0, rope_scaling=None, rope_interleave=True, scoring_func='sigmoid', attention_bias=False, attention_dropout=0.0, norm_dtype=bfloat16, gate_dtype=None, correction_bias_dtype=None, max_batch_context_length=131072, quant_config=None, dense_mlp_layers_without_quant=frozenset({}), ep_config=None, graph_mode='auto', return_logits=ReturnLogits.LAST_TOKEN, return_hidden_states=ReturnHiddenStates.NONE, eagle_aux_hidden_state_layer_ids=None, eplb_profile_enabled=False, unpadded_vocab_size=None, index_head_dim=128, index_n_heads=64, index_topk=2048, indexer_types=<factory>, indexer_rope_interleave=False, kv_b_proj_dtype=None)

source

Bases: DeepseekV3_2Config

Configuration for GLM-5.1 models.

Skeleton alias of DeepseekV3_2Config until GLM-specific bring-up diverges from DeepSeek-V3.2.

Parameters:

• dtype (DType)
• kv_params (KVCacheParamInterface)
• devices (list[DeviceRef])
• use_subgraphs (bool)
• data_parallel_degree (int)
• quantization_encoding (SupportedEncoding | None)
• vocab_size (int)
• hidden_size (int)
• intermediate_size (int)
• moe_intermediate_size (int)
• moe_layer_freq (int)
• num_hidden_layers (int)
• num_attention_heads (int)
• num_key_value_heads (int)
• n_shared_experts (int)
• n_routed_experts (int)
• routed_scaling_factor (float)
• kv_lora_rank (int)
• q_lora_rank (int)
• qk_rope_head_dim (int)
• v_head_dim (int)
• qk_nope_head_dim (int)
• topk_method (str)
• n_group (int)
• topk_group (int)
• num_experts_per_tok (int)
• first_k_dense_replace (int)
• norm_topk_prob (bool)
• hidden_act (str)
• max_position_embeddings (int)
• max_seq_len (int)
• rms_norm_eps (float)
• tie_word_embeddings (bool)
• rope_theta (float)
• rope_scaling (dict[str, Any] | None)
• rope_interleave (bool)
• scoring_func (str)
• attention_bias (bool)
• attention_dropout (float)
• norm_dtype (DType)
• gate_dtype (DType | None)
• correction_bias_dtype (DType | None)
• max_batch_context_length (int)
• quant_config (QuantConfig | None)
• dense_mlp_layers_without_quant (frozenset[int])
• ep_config (EPConfig | None)
• graph_mode (str)
• return_logits (ReturnLogits)
• return_hidden_states (ReturnHiddenStates)
• eagle_aux_hidden_state_layer_ids (list[int] | None)
• eplb_profile_enabled (bool)
• unpadded_vocab_size (int | None)
• index_head_dim (int)
• index_n_heads (int)
• index_topk (int)
• indexer_types (list[str])
• indexer_rope_interleave (bool)
• kv_b_proj_dtype (DType | None)

DEFAULT_ENCODING

DEFAULT_ENCODING: ClassVar[SupportedEncoding] = 'float8_e4m3fn'

source

SUPPORTED_ENCODINGS

SUPPORTED_ENCODINGS: ClassVar[set[SupportedEncoding]] = {'bfloat16', 'float4_e2m1fnx2', 'float8_e4m3fn'}

source

initialize()

classmethod initialize(pipeline_config, model_config=None, *, max_seq_len)

source

Initialize config, mapping GLM default RoPE to rope_scaling=None.

Parameters:

• pipeline_config (PipelineConfig)
• model_config (MAXModelConfig | None)
• max_seq_len (int)

Return type:

Self

Glm5_1Model

class max.pipelines.architectures.glm5_1.Glm5_1Model(pipeline_config, session, devices, kv_cache_config, weights, *, memory_plan, adapter=None, return_logits=ReturnLogits.ALL, return_hidden_states=ReturnHiddenStates.NONE, max_batch_size=1)

source

Bases: DeepseekV3_2Model

GLM-5.1 pipeline model.

Skeleton alias of DeepseekV3_2Model until GLM-specific bring-up diverges from DeepSeek-V3.2.

Parameters:

• pipeline_config (PipelineConfig)
• session (InferenceSession)
• devices (list[Device])
• kv_cache_config (KVCacheConfig)
• weights (Weights)
• memory_plan (MemoryPlan)
• adapter (WeightsAdapter | None)
• return_logits (ReturnLogits)
• return_hidden_states (ReturnHiddenStates)
• max_batch_size (int)

model_config_cls

model_config_cls

source

alias of Glm5_1Config

GlmReasoningParser

class max.pipelines.architectures.glm5_1.GlmReasoningParser(think_start_token_id, think_end_token_id, tool_call_start_token_id=None)

source

Bases: ReasoningParser

GLM-4.5+ (GLM-5.1 / GLM-5.2) reasoning parser for <think> sections.

GLM’s chat template appends <think> to every assistant turn via add_generation_prompt when thinking is enabled (the default), so reasoning begins implicitly without an explicit <think> token in the model output stream. Reasoning ends explicitly at </think>, or implicitly when a tool call begins (<tool_call>) — the tool-call marker is left in the content region for the tool parser to consume.

Mirrors the Qwen 3.5 parser (same <think> prefill semantics); only the delimiter tokens differ.

Parameters:

• think_start_token_id (int)
• think_end_token_id (int)
• tool_call_start_token_id (int | None)

REASONING_END

REASONING_END: ClassVar[str] = '</think>'

source

Text delimiter that closes a reasoning span (e.g. "</think>").

See REASONING_START for the declaration contract.

REASONING_START

REASONING_START: ClassVar[str] = '<think>'

source

Text delimiter that opens a reasoning span (e.g. "<think>").

Subclasses declare their delimiters here and resolve token ids from them, so each model’s delimiters are written down exactly once. Consumers that work in the text domain rather than the token domain read them from here instead of restating them.

Declare both delimiters or neither. None means this parser has no text form at all, so a text-domain consumer cannot bound a reasoning span and will leave reasoning in the assistant’s content; declaring only one is a bug, since a span needs both ends. A parser whose chat template prefills the opening delimiter still declares it – whether a given turn emits it is a property of the request, not of the parser.

from_tokenizer()

async classmethod from_tokenizer(tokenizer)

source

Construct a reasoning parser from a tokenizer.

Parameters:

tokenizer (PipelineTokenizer[Any, Any, Any])

Return type:

GlmReasoningParser

reasoning_end_token_id()

async classmethod reasoning_end_token_id(tokenizer)

source

Returns the </think> token id that closes a reasoning span.

Parameters:

tokenizer (PipelineTokenizer[Any, Any, Any])

Return type:

int | None

stream()

stream(delta_token_ids, is_currently_reasoning=True)

source

Identify a reasoning span within a streaming delta chunk.

Parameters:

• delta_token_ids (Sequence[int])
• is_currently_reasoning (bool)

Return type:

ParsedReasoningDelta

will_reason_after_prompt()

will_reason_after_prompt(prompt_token_ids)

source

Decide whether the next generated token continues a reasoning span.

The chat template embeds a literal <tool_call> example in the tool instructions, and <tool_call> is a reasoning-end delimiter, so a left-to-right scan would falsely conclude reasoning already ended. Multi-turn prompts also carry <think>/</think> from prior turns. Scan right-to-left: the last delimiter before generation is the chat template’s prefilled <think>.

Parameters:

prompt_token_ids (Sequence[int])

Return type:

bool

GlmTokenizer

class max.pipelines.architectures.glm5_1.GlmTokenizer(model_path, pipeline_config, *, revision=None, max_length=None, trust_remote_code=False, enable_llama_whitespace_fix=False, chat_template=None, **unused_kwargs)

source

Bases: TextTokenizer

Text tokenizer for GLM-4.5+ (GLM-5.1 / GLM-5.2 / GLM-5.3).

Overridden to apply reasoning parsing normalization to the chat template, and remap reasoning effort to GLM’s template.

Parameters:

• model_path (str)
• pipeline_config (PipelineConfig)
• revision (str | None)
• max_length (int | None)
• trust_remote_code (bool)
• enable_llama_whitespace_fix (bool)
• chat_template (str | None)
• unused_kwargs (Any)

apply_chat_template()

apply_chat_template(messages, tools, **chat_template_options)

source

Applies the GLM chat template, first settling GLM’s own defaults.

Parameters:

• messages (list[TextGenerationRequestMessage])
• tools (list[TextGenerationRequestTool] | None)
• chat_template_options (Any)

Return type:

str

reasoning_end_token_id

property reasoning_end_token_id: int

source

Token id of </think> (closes a GLM reasoning span).

reasoning_start_token_id

property reasoning_start_token_id: int

source

Token id of <think> (opens a GLM reasoning span).

GlmToolParser

class max.pipelines.architectures.glm5_1.GlmToolParser

source

Bases: StructuralTagToolParser

Parses GLM-4.5+ (GLM-5.1 / GLM-5.2) tool calls.

Flat layout: only CALL_BEGIN/CALL_END are set, so the base class scans for <tool_call> … </tool_call> pairs directly. Within each call the function name precedes the first <arg_key>; the remainder is parameter XML that we convert to growing JSON for streaming.

CALL_BEGIN

CALL_BEGIN: ClassVar[str] = '<tool_call>'

source

CALL_END

CALL_END: ClassVar[str] = '</tool_call>'

source

XGRAMMAR_FORMAT

XGRAMMAR_FORMAT = 'glm_4_7'

source

coerce_arguments()

coerce_arguments(args, schema)

source

Coerce parsed arguments toward their tool parameters schema.

GLM’s bare string encoding is type-ambiguous (123 is the string "123" for a string-typed field, but decodes to the integer 123). The router invokes this on the non-streaming complete parse with the tool’s parameters schema so a string field the grammar constrained is reported as the string it constrained.

Parameters:

• args (dict[str, Any])
• schema (dict[str, Any])

Return type:

dict[str, Any]

generate_tool_call_grammar()

static generate_tool_call_grammar(response_format_schema=None, tools=None, tokenizer=None, backend='xgrammar', tool_choice=None, **kwargs)

source

Generates a tool-call constrained-decoding grammar for GLM.

Returns a serialized xgrammar StructuralTag. It frames the <tool_call>func<arg_key>k</arg_key><arg_value>v</arg_value></tool_call> envelope and constrains each call’s arguments to that tool’s JSON schema using xgrammar’s native glm_xml style (bare string values, JSON for every other type). When response_format_schema is provided the tag also accepts a schema-conforming JSON response as an alternative to a tool call (mirroring the gemma4/kimi xgrammar paths).

Parameters:

• response_format_schema (dict[str, Any] | None) – Optional JSON schema dict. When provided, the grammar also accepts a JSON response matching the schema.
• tools (list[dict[str, Any]] | None) – Optional list of OpenAI-style tool dicts. None accepts any tool name.
• tokenizer (PipelineTokenizer[Any, Any, Any] | None) – Unused (the xgrammar tag references literal markers).
• backend (str) – Structured-output backend; must be "xgrammar".
• tool_choice (str | dict[str, Any] | None) – "auto", "required", or a named choice.
• **kwargs (Any) – Ignored (accepts future kwargs).

Returns:

The StructuralTag serialized as a JSON string.

Return type:

str

reset()

reset()

source

Resets internal state for a new streaming session.

Return type:

None

set_streaming_tool_schemas()

set_streaming_tool_schemas(schemas)

source

Stores per-tool parameters schemas for streaming coercion.

Parameters:

schemas (Mapping[str, dict[str, Any]])

Return type:

None