For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

mla_decode_dispatch

`comptime` values

comptime logger = Logger(stdout, prefix=String(""), source_location=False)

comptime MAX_FOLD_Q = 8

comptime MIN_FOLD_Q = 2

MLADispatchScalarArgs: Pre-computed MLA decode args for the legacy (non-capturable) path.

compute_mla_dispatch_scalars: Pure computation of the packed MLA dispatch metadata.
compute_mla_dispatch_scalars_runtime:
launch_mla_sm100_decode_enqueue_kernel:
launch_mla_sm100_decode_fp8_per_token_scale_rope_aware: Launch the FP8 per-token-scale rope-aware MLA decode kernel with split content/rope TMAs.
launch_mla_sm100_decode_native_fp8: Launch the native FP8 MLA decode kernel with FP8 Q TMA.
launch_mla_sm100_decode_native_fp8_layout_g: Launch the Layout G native FP8 MLA decode kernel (BM=32, 5-stage).
launch_mla_sm100_decode_sparse: Launch the sparse MLA decode kernel with gather4 TMA descriptors.
launch_mla_sm100_decode_sparse_kv_bf16: Launches the all-BF16 sparse MLA decode kernel.
launch_mla_sm100_decode_sparse_kv_fp8: Launches the all-FP8 sparse MLA decode kernel.
launch_mla_sm100_decode_sparse_qkv_fp8: Launches the native FP8 sparse MLA decode kernel (3 warpgroups).
mla_decode_sm100_dispatch:
mla_decode_sm100_sink_split_k: