For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_mla_store_ragged_paged_nvfp4
struct Struct_mla_store_ragged_paged_nvfp4
Registers the attention.mla.store.ragged.paged.nvfp4 graph op with the graph compiler.
The scaled counterpart of attention.mla.store.ragged.paged. A packed
NVFP4 latent carries a scale pool beside its values, and the unscaled
registration has no operands to take one, so the two are separate
registrations rather than one with optional inputs. The cache operands
arrive in the order flatten_without_attention_dispatch_metadata emits
for quantized cache params.
Implemented traits
Methods
execute
static def execute[dcp_size: Int = Int(1), dcp_interleave: Int = Int(1)](kv: T, input_row_offsets: T, freqs_cis: T, kv_norm_gamma: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, kv_scales: T, scales_page_stride: T, scales_lookup_table: T, layer_idx: UInt32, epsilon: Float32, dcp_rank: Int32, context: DeviceContext)
Stores raw MLA latent rows into a packed NVFP4 paged cache.
Behaves as attention.mla.store.ragged.paged, writing each row of
kv normed, roped and quantized at its absolute position
cache_length + token_idx, but against a cache whose rows are packed
e2m1 bytes with a per-group scale. The row's trailing rope dims stay
unscaled FP8, so a row's logical width is twice its scaled prefix
plus that tail.
Args:
- kv (
T): Raw (un-normalized) latent rows,[total_seq_len, kv_lora_rank + qk_rope_head_dim], bfloat16. - input_row_offsets (
T): Row offsets per request,[batch + 1]. - freqs_cis (
T): Interleaved RoPE table,[max_position, qk_rope_head_dim]; rowcache_length + token_idxropes a row. - kv_norm_gamma (
T): RMSNorm gamma,[kv_lora_rank]. - kv_blocks (
T): The paged cache's packed blocks,[pages, 1, layers, page_size, 1, packed_row_bytes]. - page_stride (
T): Page-to-page distance in elements; -1 when packed. - cache_lengths (
T): Cached tokens per request,[batch]. - kv_lookup_table (
T): Page lookup table,[batch, pages_per_seq]. - max_prompt_length (
T): The batch's max prompt length,[1]. - max_cache_length (
T): The batch's max cache length,[1]. - kv_scales (
T): The scale pool,kv_blocks's shape with its last dimension the scales per row. - scales_page_stride (
T): Page-to-page distance ofkv_scales, which pads independently of the values. - scales_lookup_table (
T): Page lookup table of the scales. - layer_idx (
UInt32): Layer of the cache to write. - epsilon (
Float32): RMSNorm epsilon. - dcp_rank (
Int32): This rank's index in the context-parallel group, a host scalar. Outside a group (dcp_size == 1) it is 0 and the layout never reads it. - context (
DeviceContext): Device context for GPU execution.