For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

VerifyWidthRange

VerifyWidthRange

class max.pipelines.speculative.VerifyWidthRange(*, batch_start, batch_end, num_tokens)

source

Bases: BaseModel

One inclusive decode-batch-size range and the drafts to verify in it.

Parameters:

• batch_start (int)
• batch_end (int)
• num_tokens (int)

batch_end

batch_end: int

source

Last decode batch size this range covers, inclusive.

batch_start

batch_start: int

source

First decode batch size this range covers, inclusive.

model_config

model_config: ClassVar[ConfigDict] = {}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

num_tokens

num_tokens: int

source

How many of the carried drafts the target verifies in this range.