IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

minp_max_threshold_kernel

def minp_max_threshold_kernel[dtype: DType, out_idx_type: DType, block_size: Int](min_p_coeffs_: ImmPointer[Scalar[dtype], ImmUntrackedOrigin], probs_: ImmPointer[Scalar[dtype], ImmUntrackedOrigin], out_token_ids: MutPointer[Scalar[out_idx_type], MutUntrackedOrigin], vocab_size: Int32)

Min-p sampling with no ordering anywhere.

One block per row: reduce the row's maximum probability, form the min-p threshold from it, mask and renormalize, then draw from the masked set.

The sorted path reaches the same answer in four steps that this one collapses into two scans of the row plus a per-thread walk:

  1. The row maximum. topk_wrapper[is_top_p=False] reduces the same buffer with K=1 and writes p_thresholds[batch_id] = min_p * max in dtype. A max reduce computes the same value, so the threshold below is the same number the sorted path compares against, bit for bit.
  2. Mask and renormalize: the set is p >= threshold, and the sum over it is the normalizer.
  3. Draw: r = uniform * sum, then walk the set subtracting probabilities until r <= 0. The sorted path walks the set in descending-probability order; this walks it in thread-major order, a fixed permutation of the row, which draws from the same distribution.

Parameters:

  • ​dtype (DType): DType - scalar values dtype.
  • ​out_idx_type (DType): DType - output index type.
  • ​block_size (Int): Int - Number of threads per block.

Args:

Was this page helpful?