For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

toppminp

Provides CPU implementations of top-p (nucleus) and min-p sampling for autoregressive token generation.

Functions

• ​merge: Merge two sorted subarrays into one sorted array.
• ​merge_sort_recursive: Recursive merge sort implementation.
• ​min_p_sampling: Naive CPU implementation of Min-P sampling for token selection. This function applies temperature scaling, softmax, a merge sort, and then samples tokens based on the calculated probability threshold (Min-P).
• ​sort_buf_descending: Sort each batch separately in descending order using parallel merge sort.
• ​top_p_sampling: Naive CPU implementation of Top-P sampling for token selection. This function applies temperature scaling, softmax, a merge sort, and then samples tokens based on the cumulative probability mass (Top-P).