For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
dspark_draft
The DSpark draft sampler: one fused pass per draft position.
Draws each row's draft token by Gumbel-max over the tempered softmax of
(logit + bias) / T, and writes that softmax back as q. The verifier's
rejection test is exact for any distribution the draft really samples from,
so one pass keeps the draw and the q the verifier reads from ever
disagreeing.
One thread-block cluster per row: each CTA stages its contiguous slice in
shared memory, and the row max, the exponential sum and the noised argmax
combine over the cluster's distributed shared memory. The row is read once
from global memory and q is written once. The Gumbel noise is keyed by
the row's seed and the element's index only, so a row draws the same token
under any split of the batch or of the row.
Functionsโ
- โ
dspark_draft_sample_gpu: Dispatches the fused DSpark draft sample. - โ
DSparkDraftSampleClusterKernel: One draft position's draw and its proposal distribution.