IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo package

gpu

GPU multi-head attention (MHA), cross-attention, and multi-head latent attention (MLA) kernels. Vendor-specific implementations live under amd/ and nvidia/.

Packages​

  • ​amd_rdna: TileTensor-native attention kernels for AMD RDNA3+ (gfx11xx/gfx12xx).
  • ​amd_structured: TileTensor-native attention kernels for AMD gfx950 (MI355X).
  • ​apple: Apple (Metal) GPU attention kernels.
  • ​nvidia: NVIDIA GPU attention kernels and tile-scheduling utilities.

Modules​

  • ​cp_merge_topk: Merges a decode context-parallel group's local indexer top-k selections.
  • ​mha: GPU flash-attention kernels and dispatch logic for prefill and decode.
  • ​mha_cross: Implements a naive GPU multihead cross attention kernel supporting ragged batched inputs and a paged KV cache.
  • ​mha_decode_partition_heuristic: Split-K partition-count heuristics for MHA and MLA decode kernels.
  • ​mla: GPU kernels for Multi-head Latent Attention (MLA) decoding and prefill.
  • ​mla_common: MLA types shared by every GPU vendor's decode path.
  • ​mla_graph: Provides manually fused GPU graph kernels for Multi-head Latent Attention (MLA).
  • ​mla_index_fp8: MLA FP8 index kernel for computing attention scores with paged KV cache.
  • ​mla_index_kpool: K-pool compression for the DSA indexer.
  • ​sparse_index_fp8_sm100: SM100 (B200) tensor-core FP8 MLA lightning-indexer score kernel.
  • ​sparse_index_fp8_sm100_prefill: SM100 (B200) warp-specialized PREFILL variant of the FP8 MLA indexer scorer.
  • ​sparse_indexer_common: Shared device functions for the MiniMax-M3 sparse-attention (MSA) indexer.

Was this page helpful?