IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

amd_mla_prefill_num_splits

def amd_mla_prefill_num_splits(num_heads: Int, block_m: Int, max_prompt_len: Int, batch_size: Int, num_keys: Int) -> Int

Returns the KV split count for the AMD MLA prefill grid.

A split divides the per-CTA key range evenly, which needs a cached prefix at least as long as the new tokens. Without a prefix the causal triangle leaves the early query tiles few key tiles per split, so the fp32 partials and the combine are not earned back by the extra CTAs.

When the unsplit grid already fills one residency generation (AMD_MLA_PREFILL_RESIDENT_CTAS), a split removes only the tail wave, so every split must keep AMD_MLA_PREFILL_FULL_GRID_SPLIT_MIN_KEYS keys for its fp32 partial and the combine pass to be worth a launch (no split below 32K keys). Smaller grids use the shorter AMD_MLA_PREFILL_SPLIT_MIN_KEYS bound.

Args:

  • ​num_heads (Int): Query heads per device.
  • ​block_m (Int): Query rows per CTA.
  • ​max_prompt_len (Int): Longest query span in the batch.
  • ​batch_size (Int): Number of sequences.
  • ​num_keys (Int): Total keys across the batch.

Returns:

Int: The number of KV splits, 1 when the unsplit grid is kept.

Was this page helpful?