For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
ToolParser
ToolParser
class max.pipelines.modeling.types.ToolParser(*args, **kwargs)
Bases: Protocol
Protocol for parsing tool calls from model responses.
Implementations parse model-specific tool calling formats into generic tool call structures. Supports both complete (non-streaming) and incremental (streaming) parsing modes.
Different model architectures use different tool calling formats:
- Llama models use JSON-based tool calls.
- Kimi K2.5 uses structural tags like
<|tool_call_begin|>.
The serving layer is responsible for translating these generic types to API-specific schemas (for example, OpenAI).
parse_complete()
parse_complete(response)
Parses a complete response into tool calls.
-
Parameters:
-
response (str) – The full model response text.
-
Returns:
-
A ParsedToolResponse containing any text content and parsed tool calls.
-
Raises:
-
ValueError – If the response cannot be parsed.
-
Return type:
parse_delta()
parse_delta(delta)
Parses an incremental token delta for streaming tool calls.
Accumulates tokens internally and returns tool call deltas when complete or partial tool calls can be extracted.
-
Parameters:
-
delta (str) – The incremental token(s) to process.
-
Returns:
-
- A non-empty list of
ParsedToolCallDeltawhen new content (tool name, id, or argument bytes) is ready to stream. - An empty list
[]when the parser has consumed the token and is inside the tool-calls section but has no deltas to emit yet; the caller must suppress the raw token from flowing as text content. Nonewhen more tokens are needed before anything can be emitted (e.g. buffering a potential section-begin marker).
- A non-empty list of
-
Return type:
-
list[ParsedToolCallDelta] | None
The empty-list suppression state is currently only implemented by
KimiToolParser.
Other model tool parsers return only non-empty lists or None
and should be updated to adopt this convention.
reset()
reset()
Resets internal state for a new streaming session.
-
Return type:
-
None
set_streaming_tool_schemas()
set_streaming_tool_schemas(schemas)
Provides per-tool parameter schemas for schema-driven arg streaming.
The router injects each tool’s JSON-schema parameters (keyed by
tool name) before streaming begins, letting a parser make schema-driven
decisions about how to emit argument bytes incrementally.
- Override this (schema required) for parsers whose models emit
argument values as type-ambiguous tag bodies (XML-style
<name>value</name>, where a bare scalar’s type is not recoverable from the wire —<n>42</n>could be the integer42or the string"42"). The schema decides which parameters stream character-by-character (strings) versus are buffered and typed when the element closes (numbers, booleans, …). This matters only on the streaming path — argument coercion (coerce_arguments) runs only on the non-streaming complete parse — so the schema is the sole type signal available at the moment the parser must commit bytes to the wire, and the emitted value must stay a monotonically-growing valid-JSON prefix. - Leave the null implementation (no schema needed) for raw-JSON argument formats: the JSON already carries types and streams incrementally through the base byte-diffing.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!