For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
RecurrentLeafInputs
RecurrentLeafInputs
class max.nn.kv_cache.RecurrentLeafInputs(pool, live_row_ids)
Bases: Generic[_Tensor, _Buffer]
One state leaf’s graph inputs on one device.
-
Parameters:
-
- pool (_Buffer)
- live_row_ids (_Tensor)
live_row_id()
live_row_id(layer)
Returns the [batch_size] pool row this layer runs in.
-
Parameters:
-
layer (int)
-
Return type:
live_row_ids
live_row_ids: _Tensor
pool
pool: _Buffer
Shared by all layers, so it can exceed 2**32 elements. Kernels must index it with 64-bit offsets.