For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

verify_width_operand

verify_width_operand()

max.nn.state_space.verify_width_operand(num_draft_tokens)

source

Returns the operand the speculative state ops read the verify width from.

The ops read only its length, which is host metadata, so choosing between landing the state on the forward and deferring it to the rollback costs no device synchronization. At a width of zero there is no draft to reject.

Parameters:

num_draft_tokens (int | str | Dim | integer | TypedAttr) – The graph’s draft width K, as a dim.

Returns:

A [K] int64 tensor on the host.

Return type:

TensorValue