For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python function
max_pages_per_request
max_pages_per_request()
max.nn.kv_cache.max_pages_per_request(params, max_seq_len)
Returns the most KV pages one request can address.
This is the width of a request’s row in the page lookup table: its longest sequence plus the speculative-decode slack, in pages.
-
Parameters:
-
- params (KVCacheParamInterface) – The KV cache parameters.
- max_seq_len (int) – The longest request, before speculative slack.
-
Returns:
-
The page count.
-
Return type: