For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mla_sparse_prefill_on_decode_kernel
def mla_sparse_prefill_on_decode_kernel[q_dtype: DType, cache_dtype: DType, num_heads: Int, fp8_q: Bool = True, fold_shared_index: Bool = False, has_extra_k: Bool = False, cache_is_nvfp4: Bool = False]() -> Bool
Returns whether sparse prefill of this shape runs the decode kernel.
FP8 and packed NVFP4 caches do at 8 to 64 heads (a multiple of 8) and at
128, unless the prefill folds a shared index or appends extra keys. A
packed NVFP4 cache whose context fits nvfp4_prefill_converts_once
converts once and runs the FP8 copy; otherwise the kernel gathers the
packed value and scale rows directly. Graph dispatch and sparse-index remap
preparation share this predicate. Bit-exact FP8/NVFP4 comparisons require
matching key tiles, CTA grouping, and split configuration.
Parameters:
- q_dtype (
DType): Element type of the query tensor. - cache_dtype (
DType): Element type of the latent KV cache. - num_heads (
Int): Number of query attention heads. - fp8_q (
Bool): Whether the decode branch stages Q in the FP8 cache dtype (defaults toTrue); seemla_decode_branch_fp8. - fold_shared_index (
Bool): Whether the decode branch folds a shared top-k row, which prefill cannot take (defaults toFalse). - has_extra_k (
Bool): Whether an extra-KV segment is supplied, which the masked-tile skip does not handle (defaults toFalse). - cache_is_nvfp4 (
Bool): Whethercache_dtypeis a packed NVFP4 cache (uint8with E4M3 scales), which the dtype alone cannot say (defaults toFalse). Callers that hold the cache type pass_mla_nvfp4_cache; the remap helper, which only needs the answer for logical indices that NVFP4 ops never supply, keeps the default.
Returns:
Bool: True when the sparse decode kernel serves this shape's prefill.