For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo trait
KVCollectionT
Trait for a pair of caches (keys and values).
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
Movable
comptime members
CacheType
comptime CacheType
dtype
comptime dtype
kv_params
comptime kv_params
name_str
comptime name_str
Required methods
get_key_cache
def get_key_cache(self, layer_idx: Int) -> Self.CacheType
Returns:
_Self.CacheType
get_value_cache
def get_value_cache(self, layer_idx: Int) -> Self.CacheType
Returns:
_Self.CacheType
cache_length
rows_view
def rows_view(self, first_row: Int, num_rows: Int, max_seq_length: UInt32, max_cache_length: UInt32) -> Self
Returns this collection restricted to batch rows [first_row, first_row + num_rows), with the given max lengths.
The blocks are shared, not copied.
Returns:
_Self