IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

CarryDimNames

CarryDimNames​

class max.pipelines.speculative.driver.CarryDimNames(prefix='', per_device=False)

source

Bases: object

How the driver names the carried hidden state’s batch dim per step.

Parameters:

per_device​

per_device: bool = False

source

Whether the name also carries the device index.

Naming one dim for every device asserts the per-device carries have equal batch, which holds under pure TP and not under DP. A draft that can run under DP must name them apart.

prefix​

prefix: str = ''

source

Names the carry dims {prefix}{step}_batch.

Empty keeps the batch’s own batch_size dim, which is what a draft whose step output is already batch-shaped wants.