For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
CarryDimNames
CarryDimNames
class max.pipelines.speculative.driver.CarryDimNames(prefix='', per_device=False)
Bases: object
How the driver names the carried hidden state’s batch dim per step.
per_device
per_device: bool = False
Whether the name also carries the device index.
Naming one dim for every device asserts the per-device carries have equal batch, which holds under pure TP and not under DP. A draft that can run under DP must name them apart.
prefix
prefix: str = ''
Names the carry dims {prefix}{step}_batch.
Empty keeps the batch’s own batch_size dim, which is what a draft
whose step output is already batch-shaped wants.