For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

resolve_eos_token_ids

resolve_eos_token_ids()

max.pipelines.lib.resolve_eos_token_ids(tokenizer_eos_token_id, pipeline_config)

source

Returns every token id that ends generation for the model.

Unions the tokenizer’s EOS token with the eos_token_id of the target model’s config.json, the draft model’s config.json and the target model’s generation_config.json. Each eos_token_id may be one id, a list of ids or unset. Hugging Face generate stops on the generation_config.json list, which often names more tokens than config.json (for example a chat turn terminator).

Parameters:

• tokenizer_eos_token_id (int | None) – The tokenizer’s eos_token_id, if any.
• pipeline_config (PipelineConfig | None) – The pipeline configuration to read model configs from, or None to use only the tokenizer’s EOS token.

Returns:

The set of EOS token ids.

Return type:

set[int]