For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo module
kv_cache
Registers KV-cache graph ops backed by the kv_cache and nn.kv_cache kernels.
comptime values
logger
comptime logger = Logger(stdout, prefix=String(""), source_location=False)
Structs
-
Struct_fused_qk_rms_norm_ragged_paged: Registers thekv_cache.fused_qk_rms_norm.ragged.pagedgraph op with the graph compiler. -
Struct_fused_qk_rms_norm_rope_ragged_paged: Registers thekv_cache.fused_qk_rms_norm_rope.ragged.pagedgraph op with the graph compiler. -
Struct_fused_qk_rms_norm_rope_ragged_paged_dual: Registerskv_cache.fused_qk_rms_norm_rope.ragged.paged.dualwith the graph compiler. -
Struct_fused_qk_rms_norm_rope_ragged_paged_dual_nvfp4: Registerskv_cache.fused_qk_rms_norm_rope.ragged.paged.dual.nvfp4with the graph compiler. -
Struct_fused_qk_rms_norm_rope_ragged_paged_dual_nvfp4_nvfp4_index: Registerskv_cache.fused_qk_rms_norm_rope.ragged.paged.dual.nvfp4.nvfp4_indexwith the graph compiler. -
Struct_indexer_k_quantize_store_fp8_paged: Registersindexer.k.quantize_store.fp8.paged.ragged. -
Struct_indexer_q_ssmax_rope_quantize_fp8: Registersindexer.q.ssmax_rope_quantize.fp8. -
Struct_k_matmul_ragged_paged: Registers thekv_cache.k_matmul.ragged.pagedgraph op with the graph compiler. -
Struct_k_matmul_ragged_paged_scale: Registers thekv_cache.k_matmul.ragged.paged.scalegraph op with the graph compiler. -
Struct_kv_cache_gather_rows_ragged_paged: Registers thekv_cache.gather_rows.ragged.pagedgraph op with the graph compiler. -
Struct_kv_cache_hisparse_gather_group_paged: Registers thekv_cache.hisparse.gather_group.pagedgraph op. -
Struct_kv_cache_hisparse_resolve_paged: Registers thekv_cache.hisparse.resolve.pagedgraph op. -
Struct_kv_cache_pcp_pack_rows_paged: Registers thekv_cache.pcp_pack_rows.pagedgraph op. -
Struct_kv_cache_pcp_pack_rows_paged_scaled: Registers thekv_cache.pcp_pack_rows.paged.scaledgraph op: thekv_cache.pcp_pack_rows.pagedop over a cache with a scale pool, whose operands arrive in the orderflatten_without_attention_dispatch_metadataemits for quantized cache params. -
Struct_kv_cache_pcp_unpack_rows_paged: Registers thekv_cache.pcp_unpack_rows.pagedgraph op. -
Struct_kv_cache_pcp_unpack_rows_paged_scaled: Registers thekv_cache.pcp_unpack_rows.paged.scaledgraph op: thekv_cache.pcp_unpack_rows.pagedop over a cache with a scale pool. -
Struct_kv_cache_ragged_paged_2m_iadd: Registers thekv_cache.ragged.paged.2m_iaddgraph op with the graph compiler. -
Struct_kv_cache_ragged_paged_radd: Registers thekv_cache.ragged.paged.raddgraph op with the graph compiler. -
Struct_kv_cache_row_offsets_ragged_paged: Registers thekv_cache.row_offsets.ragged.pagedgraph op with the graph compiler. -
Struct_kv_cache_store_k_scales_paged: Registers thekv_cache.store_k_scales.paged.raggedgraph op with the graph compiler. -
Struct_kv_cache_store_padded: Registers thekv_cache.store.paged.paddedgraph op with the graph compiler. -
Struct_kv_cache_store_paged: Registers thekv_cache.store.paged.raggedgraph op with the graph compiler. -
Struct_kv_matmul_ragged_paged: Registers thekv_cache.kv_matmul.ragged.pagedgraph op with the graph compiler. -
Struct_rms_norm_kv_cache_ragged_paged: Registers thekv_cache.rms_norm.ragged.pagedgraph op with the graph compiler. -
Struct_rms_norm_value_cache_ragged_paged: Registers thekv_cache.rms_norm_value_cache.ragged.pagedgraph op with the graph compiler.
Functions
-
fused_dual_qk_rms_norm_rope_nvfp4_from_graph: Runsfused_dual_qk_rms_norm_rope_nvfp4_ragged_pagedfrom graph operands, once the ops have built their cache collections. -
load_staged_row: Readswidthelements of one staged[token, head, col]row.