IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mla_decode_dma_warp

def mla_decode_dma_warp[q_type: DType, k_t: MHAOperand, mask_t: MHAMask, BM: Int, BN: Int, WM: Int, WN: Int, depth: Int, q_seq_len: Int]() -> Bool

Returns whether the FP8 MLA decode fold adds a DMA producer warp.

The warp-local fold (WN == BN) runs one 16-row warp per SIMD, so a CTA of at most 3 compute warps leaves a SIMD free. A producer warp there streams the next KV tile into a second LDS stage for a whole tile ahead, so the compute warps never issue or wait on a DMA of their own. The two K stages and the P tile must fit one CU's LDS.

Parameters:

  • ​q_type (DType): The query dtype.
  • ​k_t (MHAOperand): The K operand type.
  • ​mask_t (MHAMask): The mask type.
  • ​BM (Int): Rows of the CTA's M tile.
  • ​BN (Int): Keys of the KV tile.
  • ​WM (Int): Rows per warp.
  • ​WN (Int): Keys per warp.
  • ​depth (Int): The head depth.
  • ​q_seq_len (Int): Query rows per request.

Returns:

Bool: True when the launch adds the producer warp.

Was this page helpful?