For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

TopKTopPMaskedProbsKernel

def TopKTopPMaskedProbsKernel[block_size: Int, vec_size: Int, dtype: DType, LogitsLayoutType: TensorLayout, logits_origin: ImmOrigin](logits: TileTensor[dtype, LogitsLayoutType, logits_origin], probs_ptr: Pointer[Float32, MutAnyOrigin], top_k_arr: Optional[Pointer[Int64, ImmutAnyOrigin]], top_k_val: Int32, top_p_arr: Optional[Pointer[Float32, ImmutAnyOrigin]], top_p_val: Float32, temperature: Optional[Pointer[Float32, ImmutAnyOrigin]], d: Int32)

Writes each row's top-k/top-p masked softmax, without sampling.

Works in the unnormalized domain e_i = exp((logit_i - row_max) / temp): a token survives the joint constraint iff e > cutoff (recovered by the same dual-pivot search the sampler uses) and its masked probability is e / kept_mass. The output row is that masked renormalized distribution -- the same tensor TopKTopPSamplingFromProbKernel emits under emit_dist, so a verifier's target-side probabilities and a draft's proposal distribution are described identically.