For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

TextGenerationPipelineInterface

TextGenerationPipelineInterface

class max.pipelines.TextGenerationPipelineInterface

source

Bases: Pipeline[TextGenerationInputs[TextGenerationContextType], TextGenerationOutput], Generic[TextGenerationContextType]

Interface for text generation pipelines.

compiled_modules

compiled_modules: Mapping[str, CompiledCallable[..., Any]]

source

The graphs this pipeline compiled, by role, for export: the model’s, then the sampler’s.

kv_manager

abstract property kv_manager: PagedKVCacheManagerInterface

source

Returns the KV cache managers for this pipeline.

pipeline_config

abstract property pipeline_config: PipelineConfig

source

Returns the pipeline configuration.

tokenizer

abstract property tokenizer: PipelineTokenizer[TextGenerationContextType, ndarray[tuple[Any, ...], dtype[integer[Any]]], TextGenerationRequest]

source

Returns the tokenizer for this pipeline.