For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo trait

KVCollectionT

Trait for a pair of caches (keys and values).

Implemented traits

AnyType, Copyable, ImplicitlyCopyable, Movable

comptime members

CacheType

comptime CacheType

dtype

comptime dtype

kv_params

comptime kv_params

name_str

comptime name_str

Required methods

__init__

def __init__(out self, *, copy: Self)

Create a new instance of the value by copying an existing one.

Args:

• ​copy (_Self): The value to copy.

Returns:

_Self

def __init__(out self, *, deinit move: Self)

Create a new instance of the value by moving the value of another.

Args:

• ​move (_Self): The value to move.

Returns:

_Self

get_key_cache

def get_key_cache(self, layer_idx: Int) -> Self.CacheType

Returns:

_Self.CacheType

get_value_cache

def get_value_cache(self, layer_idx: Int) -> Self.CacheType

Returns:

_Self.CacheType

cache_length

def cache_length(self, bs_idx: Int) -> Int

Returns:

Int

Provided methods

copy

def copy(self) -> Self

Explicitly construct a copy of self, a convenience method for Self(copy=self) when the type is inconvenient to write out.

Overriding this method is not allowed.

Returns:

_Self: A copy of this value.