For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python module
max.pipelines.kv_cache.checkpoint
Decides where to checkpoint a request’s cache.
Checkpointer
CheckpointContext | alias of Context |
|---|---|
Checkpointer | Collects lifecycle events and runs heuristics. |
CheckpointerFactory | Builds a checkpointer once every query its context needs is provided. |
Events and checkpoints
Checkpoint | A token position of a request to persist the cache at. |
|---|---|
CheckpointEvent | alias of Event |
CheckpointEventKind | alias of EventKind |
Heuristics
CheckpointHeuristic | alias of Heuristic |
|---|
checkpoint_at_end_of_prompt | Checkpoints where a prompt ends and where a replay of it resumes. |
|---|---|
checkpoint_at_release | Checkpoints a request where it releases its cache. |