IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mla_sparse_check_topk_lengths

def mla_sparse_check_topk_lengths[cache_dtype: DType, num_heads: Int](available: Int, batch_size: Int, num_tokens: Int, max_seq_len: Int)

Raises unless a sparse prefill-decode graph op's topk_lengths covers the branch the batch takes.

The decode branch reads one length per sequence; a prefill-shaped branch, including prefill on the decode kernel, reads one per query token.

Parameters:

  • ​cache_dtype (DType): Element type of the latent KV cache.
  • ​num_heads (Int): Number of query attention heads.

Args:

  • ​available (Int): Entries in the topk_lengths buffer.
  • ​batch_size (Int): Sequences in the batch.
  • ​num_tokens (Int): Query tokens in the batch.
  • ​max_seq_len (Int): Maximum query length in the batch.

Was this page helpful?