IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mla_graph_takes_decode_branch

def mla_graph_takes_decode_branch[cache_dtype: DType, num_heads: Int, sparse_mla: Bool = False, mask_places_query_span: Bool = False](max_seq_len: Int) -> Bool

Returns whether mla_prefill_decode_graph_* takes its decode branch.

Parameters:

  • ​cache_dtype (DType): Element type of the latent KV cache.
  • ​num_heads (Int): Number of query attention heads.
  • ​sparse_mla (Bool): Whether top-k sparse attention is in use; on AMD it routes S>1 to prefill.
  • ​mask_places_query_span (Bool): Whether the batch's mask places the query span from the comptime query width (see mla_mask_places_query_span), which excludes the padded short-chunk fold (defaults to False).

Args:

  • ​max_seq_len (Int): Maximum query length in the batch.

Returns:

Bool: True when max_seq_len fits the decode branch's query fold.

Was this page helpful?