For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Python class
MHAAttnKey
MHAAttnKey
class max.nn.kv_cache.MHAAttnKey(batch_size, max_prompt_length, num_partitions)
Bases: AttnKey
Decode dispatch metadata for multi-head attention (MHA).
dispatch_metadata_spec()
classmethod dispatch_metadata_spec()
MHA decode kernels read four ints, on the host.
-
Return type:
-
DispatchMetadataSpec
pack_into()
pack_into(into, max_cache_valid_length)
Writes batch size, prompt width, partition count, cache length.