For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

AudioGenerationTokenizer

AudioGenerationTokenizer

class max.pipelines.lib.AudioGenerationTokenizer(model_path, pipeline_config, subfolder='tokenizer', *, revision=None, max_length=None, trust_remote_code=False, default_audio_duration, default_num_inference_steps, **unused_kwargs)

source

Bases: PipelineTokenizer[AudioContext, ndarray[tuple[Any, …], dtype[int64]], OpenResponsesRequest]

Turns an audio generation request into an AudioContext.

Audio checkpoints assemble their prompt from more than the request’s text – lyrics, a caption, and special tokens that mark each part – and that assembly is part of the checkpoint’s contract, so a subclass owns it via assemble_prompt(). A subclass that guides also overrides unconditional_ids() to say what its unconditional prompt is.

Parameters:

• model_path (str) – Path to the model repository.
• pipeline_config (PipelineConfig) – The resolved pipeline configuration.
• subfolder (str) – Subfolder of the repository holding the tokenizer.
• revision (str | None) – Git revision of the repository to load.
• max_length (int | None) – Maximum length of the assembled prompt, in tokens.
• trust_remote_code (bool) – Whether to run tokenizer code from the repository.
• default_audio_duration (float) – Duration in seconds to generate when the request does not ask for one. Required of the subclass: how long a checkpoint renders by default is the checkpoint’s property, and a number chosen here would silently become every model’s.
• default_num_inference_steps (int) – Denoising steps to take when the request does not ask for a count, on the same terms.
• unused_kwargs (Any)

assemble_prompt()

assemble_prompt(description, lyrics)

source

Returns the text to tokenize, in the checkpoint’s prompt format.

Parameters:

• description (str) – The request’s prompt, describing the audio to make.
• lyrics (str | None) – Lyrics to sing, if the request carried any.

Returns:

The assembled prompt.

Return type:

str

decode()

async decode(encoded, **kwargs)

source

Raises: audio generation returns samples, never text.

Parameters:

• encoded (ndarray[tuple[Any, ...], dtype[int64]])
• kwargs (Any)

Return type:

str

encode()

async encode(prompt, add_special_tokens=True)

source

Tokenizes an already-assembled prompt.

Parameters:

• prompt (str) – The assembled prompt.
• add_special_tokens (bool) – Whether the tokenizer adds its own special tokens. Assembled prompts usually carry theirs already.

Returns:

The prompt’s token ids.

Raises:

PromptTooLongError – If the prompt is longer than max_length.

Return type:

ndarray[tuple[Any, …], dtype[int64]]

eos_token_ids

property eos_token_ids: set[int]

source

Returns the end-of-sequence token ids of the prompt tokenizer.

These terminate the text prompt, not the generated audio: an audio model stops on a token of its own that the executor owns.

expects_content_wrapping

property expects_content_wrapping: bool

source

audio requests carry a prompt, not chat messages.

Type:

Returns False

new_context()

async new_context(request)

source

Builds the context for one audio generation request.

Parameters:

request (OpenResponsesRequest) – The incoming request.

Returns:

The context to generate from.

Raises:

ValueError – If the request carries no prompt.

Return type:

AudioContext

unconditional_ids()

unconditional_ids(token_ids)

source

Returns the unconditional counterpart of token_ids, or None.

None means the model does not guide, and leaves AudioContext.negative_tokens unset.

Parameters:

token_ids (ndarray[tuple[Any, ...], dtype[int64]]) – The conditional prompt’s token ids.

Return type:

ndarray[tuple[Any, …], dtype[int64]] | None