IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

DSparkDraftSampleClusterKernel

def DSparkDraftSampleClusterKernel[dtype: DType, cluster_size: Int, LogitsLayoutType: TensorLayout, logits_origin: ImmOrigin, BiasLayoutType: TensorLayout, bias_origin: ImmOrigin, DistLayoutType: TensorLayout, TokensLayoutType: TensorLayout, LogitsEngine: TensorEngine, BiasEngine: TensorEngine, DistEngine: TensorEngine, TokensEngine: TensorEngine](logits: TileTensor[dtype, LogitsLayoutType, logits_origin, Engine=LogitsEngine], bias: TileTensor[.float32, BiasLayoutType, bias_origin, Engine=BiasEngine], temperature: ImmPointer[Float32, ImmutAnyOrigin], seed: ImmPointer[UInt64, ImmutAnyOrigin], out_tokens: TileTensor[.int64, TokensLayoutType, MutAnyOrigin, Engine=TokensEngine], dist: TileTensor[.float32, DistLayoutType, MutAnyOrigin, Engine=DistEngine], d: Int32)

One draft position's draw and its proposal distribution.

Each CTA owns a contiguous slice of the row and stages x = (logit + bias) / T in shared memory. Three cluster reductions follow: the row max m, the row sum z = sum(exp(x - m)), and the noised argmax that picks the token. q = exp(x - m) / z is written from the staged slice, so the row is read from global memory once.

The launch must set cluster_dim to cluster_size.

Was this page helpful?