For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
kv_cache
comptime values
NVFP4_QK_LAUNCH_AUTO
comptime NVFP4_QK_LAUNCH_AUTO = 0
NVFP4_QK_LAUNCH_ROW
comptime NVFP4_QK_LAUNCH_ROW = 1
NVFP4_QK_LAUNCH_WARP
comptime NVFP4_QK_LAUNCH_WARP = 2
Functions
-
fused_dual_qk_rms_norm_rope_nvfp4_ragged_paged: Writes MiniMax-M3's main K/V and index K from staging, quantizing auint8cache to NVFP4. -
fused_dual_qk_rms_norm_rope_nvfp4_ragged_paged_gpu: Runs the five-band grid offused_dual_qk_rms_norm_rope_nvfp4_ragged_paged. -
fused_dual_qk_rms_norm_rope_nvfp4_ragged_paged_warp_gpu: Runs the warp launch offused_dual_qk_rms_norm_rope_nvfp4_ragged_paged. -
fused_dual_qk_rms_norm_rope_ragged_paged: Fuses twofused_qk_rms_norm_rope_ragged_pagedlaunches into one. -
fused_qk_rms_norm_ragged_paged: Applies per-head RMSNorm to Q and new K-cache entries in one GPU launch. -
fused_qk_rms_norm_rope_ragged_paged: Fuses per-head RMSNorm and RoPE for Q and new K-cache entries. -
generic_flash_attention_kv_cache_padded: -
generic_fused_qkv_matmul_kv_cache_bshd_paged: Performs a fused QKV matmul. Q outputs are written to the output argument while K and V outputs are written in-place into k_cache and v_cache. -
generic_get_paged_cache: -
generic_get_paged_cache_with_scales: Builds a scaled paged KV cache collection from graph operands. -
nvfp4_kv_store_row: Stores one (token, head) row heldsimd_widthcolumns per thread. -
nvfp4_kv_write_row: Writes one (token, head) K or V row from staging into a paged cache. -
paged_cache_rows: Builds the paged cache over batch rows[first_row, first_row + num_rows), with max lengths fromsplit_args[max_lengths_slot:+2]. -
rms_norm_kv_cache_ragged_paged: Performs RMSNorm in place on new entries in the key cache. -
rms_norm_value_cache_ragged_paged: Performs RMSNorm in place on new entries in the value cache.