For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
dspark_draft_sample_gpu
def dspark_draft_sample_gpu[dtype: DType, LogitsLayoutType: TensorLayout, logits_origin: ImmOrigin, BiasLayoutType: TensorLayout, bias_origin: ImmOrigin, TemperatureLayoutType: TensorLayout, SeedLayoutType: TensorLayout, DistLayoutType: TensorLayout, TokensLayoutType: TensorLayout, LogitsEngine: TensorEngine = DefaultEngine, BiasEngine: TensorEngine = DefaultEngine, TemperatureEngine: TensorEngine = DefaultEngine, SeedEngine: TensorEngine = DefaultEngine, DistEngine: TensorEngine = DefaultEngine, TokensEngine: TensorEngine = DefaultEngine](ctx: DeviceContext, logits: TileTensor[dtype, LogitsLayoutType, logits_origin, Engine=LogitsEngine], bias: TileTensor[.float32, BiasLayoutType, bias_origin, Engine=BiasEngine], temperature: TileTensor[.float32, TemperatureLayoutType, ImmutAnyOrigin, Engine=TemperatureEngine], seed: TileTensor[.uint64, SeedLayoutType, ImmutAnyOrigin, Engine=SeedEngine], out_tokens: TileTensor[.int64, TokensLayoutType, MutAnyOrigin, Engine=TokensEngine], dist: TileTensor[.float32, DistLayoutType, MutAnyOrigin, Engine=DistEngine])
Dispatches the fused DSpark draft sample.
Args:
- ctx (
DeviceContext): Device context. - logits (
TileTensor[dtype, LogitsLayoutType, logits_origin, Engine=LogitsEngine]): Raw head logits[rows, vocab], bfloat16 or float32. - bias (
TileTensor[.float32, BiasLayoutType, bias_origin, Engine=BiasEngine]): The Markov bias[rows, vocab], float32. - temperature (
TileTensor[.float32, TemperatureLayoutType, ImmutAnyOrigin, Engine=TemperatureEngine]): Per-row temperature[rows]; non-positive clamps. - seed (
TileTensor[.uint64, SeedLayoutType, ImmutAnyOrigin, Engine=SeedEngine]): Per-row RNG seed[rows], uint64. - out_tokens (
TileTensor[.int64, TokensLayoutType, MutAnyOrigin, Engine=TokensEngine]): Sampled column per row,int64 [rows]. - dist (
TileTensor[.float32, DistLayoutType, MutAnyOrigin, Engine=DistEngine]): The sampled distribution[rows, vocab], float32.
Raises:
Error: If the shapes disagree or a row does not fit the staging budget even at the widest cluster.