For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python function
cache_tensor
cache_tensor()
max.nn.common_layers.kv_cache.cache_tensor(shards, mapping=None)
Returns one cache input’s shards as a tensor, keeping real ones.
Like from_shard_values(), except that real
tensors, as a decoder called eagerly gets, keep their buffers, so ops
that write the cache in place write the decoder’s buffers.
-
Parameters:
-
- shards (Sequence[TensorValue | BufferValue | Tensor]) – One graph value or tensor per device.
- mapping (DeviceMapping | None) – How the shards make up the tensor. Required for more than one shard.
-
Returns:
-
A tensor backed by
shards. -
Return type: