IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

mha_decode_partition_heuristic

Split-K partition-count heuristics for MHA and MLA decode kernels.

Exposes per-backend functions that decide how many partitions the decode split-K loop should run across, plus num_keys-independent upper bounds used to launch graph-stable decode grids whose extra partitions early-return in the kernel. CUDA uses a simple SM-fill target clamped to the single-warp reducer limit; HIP (MI355X) uses a wave-aligned formula tuned for both MHA and MLA decode shapes.

comptime values​

AMD_MLA_BF16_DECODE_BLOCK_M​

comptime AMD_MLA_BF16_DECODE_BLOCK_M = 16

AMD_MLA_DECODE_FOLD_MAX_NUM_HEADS​

comptime AMD_MLA_DECODE_FOLD_MAX_NUM_HEADS = 16

AMD_MLA_DECODE_FOLD_MAX_SEQ_LEN​

comptime AMD_MLA_DECODE_FOLD_MAX_SEQ_LEN = 128

AMD_MLA_DECODE_MTP_SEQ_LEN​

comptime AMD_MLA_DECODE_MTP_SEQ_LEN = 8

MAX_HIP_PARTITIONS​

comptime MAX_HIP_PARTITIONS = 256

MLA_DECODE_KEY_TILE_ROWS​

comptime MLA_DECODE_KEY_TILE_ROWS = 128

Functions​

Was this page helpful?