IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_mla_decode_graph_bf16_paged_sparse_nvfp4_cp

struct Struct_mla_decode_graph_bf16_paged_sparse_nvfp4_cp

Registers the attention.mla.graph.decode.paged.sparse.nvfp4.cp graph op.

attention.mla.graph.decode.paged.sparse.nvfp4 for decode context parallelism; see attention.mla.graph.decode.paged.fp8.sparse.cp for the per-token lengths, the lse output, split_len_bound and the CP store. A row of sparse_indices is indices_stride / 2 value rows then as many scale-pool rows, and a token's length counts entries of each half: the compacted value rows sit at [0, len) and their scale rows at [indices_stride / 2, indices_stride / 2 + len). SM100 only.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[mask_str: StaticString, dcp_size: Int, dcp_interleave: Int, indices_stride: Int, physical_indices: Bool, q_absorbed: Bool = False, latent_output: Bool = False](output: T, lse: T, q: T, kv: T, input_row_offsets: T, freqs_cis: T, kv_norm_gamma: T, w_uk: T, w_uv: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, kv_scales: T, scales_page_stride: T, scales_lookup_table: T, layer_idx: UInt32, scale: Float32, epsilon: Float32, scalar_args: T, sparse_indices: T, sparse_lengths: T, split_len_bound: Int32, dcp_rank: Int32, context: DeviceContext)

Was this page helpful?