For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
spec_glue
The speculative-decode accept path's per-step glue, one launch each.
Both kernels cover tensors of a few dozen entries -- batch x draft positions
for the rejection mask, and one scattered pair of probabilities per drafted
token for the accept test -- where a chain of whole-tensor graph ops pays a
launch floor per member. Every value is an integer or an elementwise float
with no reordering, so each folded form is bit-identical to the chain it
replaces: only the number of dispatches changes.
Functionsโ
- โ
accept_pair: Writes the sampled-draft verdict's per-position pair, in one launch. - โ
first_rejected_index: Writes each row's first rejected draft position,stepsif none.