For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
MLAAttnKey
MLAAttnKey
class max.nn.kv_cache.MLAAttnKey(batch_size, max_prompt_length, num_partitions)
Bases: AttnKey
Decode dispatch metadata for multi-latent attention (MLA).
dispatch_metadata_spec()
classmethod dispatch_metadata_spec()
MLA decode kernels read three ints, on the accelerator.
-
Return type:
-
DispatchMetadataSpec
pack_into()
pack_into(into, max_cache_valid_length)
Writes batch size, prompt width and partition count.
The cache length is not part of the MLA dispatch buffer; it is carried
separately in max_cache_length.