IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

DraftCache

DraftCache​

class max.pipelines.speculative.driver.DraftCache(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

source

Bases: Enum

Which cache the draft writes, and so what the driver advances.

OWN​

OWN = 'own'

source

The draft has a cache of its own.

The driver advances its lengths by the accepted count before the loop and by one per step, and substitutes them into the collections.

PER_DEPTH​

PER_DEPTH = 'per_depth'

source

Each depth owns a cache and writes a single token into it.

A step’s window therefore starts empty rather than extending the one before it, so the driver zeroes the lengths instead of advancing them. Declaring the ownership is what selects that; a draft cannot ask for the zeroing and the advance at once.

TARGET​

TARGET = 'target'

source

The draft only reads the target’s cache.

It writes no KV slot, so its position stays fixed across the steps and the per-step advance is skipped. The accepted-count advance still runs, reaching the draft as SequentialBatch.draft_cache_lengths rather than being substituted into the collections, which would lengthen the window it attends over.