For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python function

batch_max_k

batch_max_k()

max.pipelines.sampling.batch_max_k(top_k)

source

Returns the max_k scalar the fused sampler takes for top_k.

The kernel resolves a row’s top_k of -1 (no limit) to max_k, so the plain batch maximum would hand that row a batch-mate’s explicit top_k: next to a greedy request it would decode greedily. Returning -1 whenever any row has no limit lets the kernel give those rows its own unrestricted bound while explicit rows keep their top_k.

Parameters:

top_k (ndarray[tuple[Any, ...], dtype[int64]]) – Per-row top_k values of the batch.

Returns:

-1 if any row has no top-k limit, else the largest top_k.

Return type:

int