For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
min_p_sampling
def min_p_sampling[dtype: DType, out_idx_type: DType, //, _test_sort: Bool = False](min_ps: TileTensor[dtype, Engine=min_ps.Engine, address_space=min_ps.address_space, linear_idx_type=min_ps.linear_idx_type], input_logits: TileTensor[dtype, Engine=input_logits.Engine, address_space=input_logits.address_space, linear_idx_type=input_logits.linear_idx_type], out_token_ids: TileTensor[out_idx_type, Engine=out_token_ids.Engine, address_space=out_token_ids.address_space, linear_idx_type=out_token_ids.linear_idx_type], temperature: Scalar[dtype] = 1)
Naive CPU implementation of Min-P sampling for token selection. This function applies temperature scaling, softmax, a merge sort, and then samples tokens based on the calculated probability threshold (Min-P).
Parameters:
- dtype (
DType): Element type ofinput_logits,min_ps, andtemperature(inferred). - out_idx_type (
DType): Element type of theout_token_idstensor (inferred). - _test_sort (
Bool): When true, copies the sorted probabilities back intoinput_logitsto verify descending order (defaults to false).
Args:
- min_ps (
TileTensor[dtype, Engine=min_ps.Engine, address_space=min_ps.address_space, linear_idx_type=min_ps.linear_idx_type]): Per-batch minimum probability thresholds in the range (0, 1). - input_logits (
TileTensor[dtype, Engine=input_logits.Engine, address_space=input_logits.address_space, linear_idx_type=input_logits.linear_idx_type]): Rank-2 logits tensor of shape (batch, vocab) read for temperature scaling and softmax. - out_token_ids (
TileTensor[out_idx_type, Engine=out_token_ids.Engine, address_space=out_token_ids.address_space, linear_idx_type=out_token_ids.linear_idx_type]): Rank-2 output tensor receiving the sampled token index at column 0 of each batch row. - temperature (
Scalar[dtype]): Positive scalar divisor applied to logits before the softmax (defaults to 1).