For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mla_decode_dma_warp
def mla_decode_dma_warp[q_type: DType, k_t: MHAOperand, mask_t: MHAMask, BM: Int, BN: Int, WM: Int, WN: Int, depth: Int, q_seq_len: Int]() -> Bool
Returns whether the FP8 MLA decode fold adds a DMA producer warp.
The warp-local fold (WN == BN) runs one 16-row warp per SIMD, so a CTA of
at most 3 compute warps leaves a SIMD free. A producer warp there streams
the next KV tile into a second LDS stage for a whole tile ahead, so the
compute warps never issue or wait on a DMA of their own. The two K stages
and the P tile must fit one CU's LDS.
Parameters:
- q_type (
DType): The query dtype. - k_t (
MHAOperand): The K operand type. - mask_t (
MHAMask): The mask type. - BM (
Int): Rows of the CTA's M tile. - BN (
Int): Keys of the KV tile. - WM (
Int): Rows per warp. - WN (
Int): Keys per warp. - depth (
Int): The head depth. - q_seq_len (
Int): Query rows per request.
Returns:
Bool: True when the launch adds the producer warp.