IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

accept_pair

def accept_pair(probs: TileTensor[.float32, Engine=probs.Engine, address_space=probs.address_space, linear_idx_type=probs.linear_idx_type], dist: TileTensor[.float32, Engine=dist.Engine, address_space=dist.address_space, linear_idx_type=dist.linear_idx_type], tokens: TileTensor[.int64, Engine=tokens.Engine, address_space=tokens.address_space, linear_idx_type=tokens.linear_idx_type], coins: TileTensor[.float32, Engine=coins.Engine, address_space=coins.address_space, linear_idx_type=coins.linear_idx_type], rejected: TileTensor[.bool, Engine=rejected.Engine, address_space=rejected.address_space, linear_idx_type=rejected.linear_idx_type], fallback: TileTensor[.int64, Engine=fallback.Engine, address_space=fallback.address_space, linear_idx_type=fallback.linear_idx_type], ctx: DeviceContext)

Writes the sampled-draft verdict's per-position pair, in one launch.

For each (request, draft position) the accept test needs two numbers at one vocabulary index -- the target's masked probability there and the proposal's probability there -- and the residual needs to know whether the row carried a proposal at all. Reaching them with graph ops costs a [rows, steps, 3] index tensor to gather through, then a compare, a select, a multiply and a compare: one launch per member over a mask of a few dozen entries.

q == 0 means the scheduler had no distribution for the row, so the proposal mass is taken as one (the test degrades to typical acceptance) and the residual subtracts a one-hot at the drafted token instead of the proposal. Both the substitution and that per-row choice happen here, so the caller reads only the two decisions.

Args:

Raises:

Error: If a rank or a vocabulary size disagrees with probs.

Was this page helpful?