IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

mla

GPU kernels for Multi-head Latent Attention (MLA) decoding and prefill.

Provides the flare_mla_decoding and flare_mla_prefill entrypoints plus their platform-specific dispatch and kernel implementations targeting NVIDIA (SM80/SM100) and AMD (gfx950) GPUs, including split-K reduction, multi-token prediction (MTP) query folding, per-token scale, and sparse-attention support.

comptime values​

AMD_MLA_DECODE_FOLD_M_MAX​

comptime AMD_MLA_DECODE_FOLD_M_MAX = 2048

AMD_MLA_PREFILL_COMBINE_WARPS​

comptime AMD_MLA_PREFILL_COMBINE_WARPS = 4

AMD_MLA_PREFILL_FULL_GRID_SPLIT_MIN_KEYS​

comptime AMD_MLA_PREFILL_FULL_GRID_SPLIT_MIN_KEYS = 16384

AMD_MLA_PREFILL_MAX_SPLITS​

comptime AMD_MLA_PREFILL_MAX_SPLITS = 32

AMD_MLA_PREFILL_RESIDENT_CTAS​

comptime AMD_MLA_PREFILL_RESIDENT_CTAS = 512

AMD_MLA_PREFILL_SPLIT_MIN_KEYS​

comptime AMD_MLA_PREFILL_SPLIT_MIN_KEYS = 1024

AMD_MLA_PREFILL_SPLIT_TARGET_CTAS​

comptime AMD_MLA_PREFILL_SPLIT_TARGET_CTAS = 1024

MLA_DECODE_FOLD_MAX_SEQ_LEN​

comptime MLA_DECODE_FOLD_MAX_SEQ_LEN = AMD_MLA_DECODE_FOLD_MAX_SEQ_LEN

MLA_DECODE_MAX_SEQ_LEN​

comptime MLA_DECODE_MAX_SEQ_LEN = AMD_MLA_DECODE_MTP_SEQ_LEN

Structs​

Traits​

Functions​

Was this page helpful?