For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mla_graph_takes_decode_branch
def mla_graph_takes_decode_branch[cache_dtype: DType, num_heads: Int, sparse_mla: Bool = False, mask_places_query_span: Bool = False](max_seq_len: Int) -> Bool
Returns whether mla_prefill_decode_graph_* takes its decode branch.
Parameters:
- cache_dtype (
DType): Element type of the latent KV cache. - num_heads (
Int): Number of query attention heads. - sparse_mla (
Bool): Whether top-k sparse attention is in use; on AMD it routes S>1 to prefill. - mask_places_query_span (
Bool): Whether the batch's mask places the query span from the comptime query width (seemla_mask_places_query_span), which excludes the padded short-chunk fold (defaults toFalse).
Args:
- max_seq_len (
Int): Maximum query length in the batch.
Returns:
Bool: True when max_seq_len fits the decode branch's query fold.