For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

MSAAttnKey

MSAAttnKey

class max.nn.kv_cache.MSAAttnKey

source

Bases: AttnKeyInterface

Decode dispatch metadata for multi-step attention (MSA).

dispatch_metadata_spec()

classmethod dispatch_metadata_spec()

source

A single host int, since MSA kernels read no dispatch metadata.

Return type:

DispatchMetadataSpec

pack_into()

pack_into(into, max_cache_valid_length)

source

Writes the sentinel the placeholder input carries.

Parameters:

• into (ndarray)
• max_cache_valid_length (int)

Return type:

None