For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
minp_max_threshold_kernel
def minp_max_threshold_kernel[dtype: DType, out_idx_type: DType, block_size: Int](min_p_coeffs_: ImmPointer[Scalar[dtype], ImmUntrackedOrigin], probs_: ImmPointer[Scalar[dtype], ImmUntrackedOrigin], out_token_ids: MutPointer[Scalar[out_idx_type], MutUntrackedOrigin], vocab_size: Int32)
Min-p sampling with no ordering anywhere.
One block per row: reduce the row's maximum probability, form the min-p threshold from it, mask and renormalize, then draw from the masked set.
The sorted path reaches the same answer in four steps that this one collapses into two scans of the row plus a per-thread walk:
- The row maximum.
topk_wrapper[is_top_p=False]reduces the same buffer with K=1 and writesp_thresholds[batch_id] = min_p * maxindtype. A max reduce computes the same value, so the threshold below is the same number the sorted path compares against, bit for bit. - Mask and renormalize: the set is
p >= threshold, and the sum over it is the normalizer. - Draw:
r = uniform * sum, then walk the set subtracting probabilities untilr <= 0. The sorted path walks the set in descending-probability order; this walks it in thread-major order, a fixed permutation of the row, which draws from the same distribution.
Parameters:
- dtype (
DType): DType - scalar values dtype. - out_idx_type (
DType): DType - output index type. - block_size (
Int): Int - Number of threads per block.
Args:
- min_p_coeffs_ (
ImmPointer[Scalar[dtype], ImmUntrackedOrigin]): Per-row min-p coefficients, in the caller's range [0, 1] (min_p_sampling_gpu'smin_ps), not probabilities. - probs_ (
ImmPointer[Scalar[dtype], ImmUntrackedOrigin]): Post-softmax probabilities [batch_size, vocab_size], the buffertopp_minp_sampling_kernelreads. - out_token_ids (
MutPointer[Scalar[out_idx_type], MutUntrackedOrigin]): Output token ids [batch_size]. - vocab_size (
Int32): Number of tokens in the vocabulary per row.