IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

spec_glue

The speculative-decode accept path's per-step glue, one launch each.

Both kernels cover tensors of a few dozen entries -- batch x draft positions for the rejection mask, and one scattered pair of probabilities per drafted token for the accept test -- where a chain of whole-tensor graph ops pays a launch floor per member. Every value is an integer or an elementwise float with no reordering, so each folded form is bit-identical to the chain it replaces: only the number of dispatches changes.

Functionsโ€‹

Was this page helpful?