For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

ChatEncoderOutcomesProbe

ChatEncoderOutcomesProbe

class max.pipelines.modeling.types.ChatEncoderOutcomesProbe(*args, **kwargs)

source

Bases: Protocol

Optional tokenizer capability: report how its chat prompts encoded.

A tokenizer that can encode chats with a --tokenizer-impl encoder implements this so the server can count the chats that encoder served and the ones that fell back to HuggingFace. A separate protocol for the reasons given on PreprocessCacheStatsProbe.

take_chat_encoder_outcomes()

take_chat_encoder_outcomes()

source

Returns the chats encoded since the last call, and forgets them.

Must be cheap: the server calls it per request on the event loop.

Returns:

custom for a chat the encoder served, or fallback for one it failed on, which HuggingFace encoded instead.

Return type:

A count per outcome