IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_mla_decode_graph_bf16_paged_sparse_cp

struct Struct_mla_decode_graph_bf16_paged_sparse_cp

Registers the attention.mla.graph.decode.paged.sparse.cp graph op.

attention.mla.graph.decode.paged.sparse (BF16 weights, BF16 or FP8 latent cache) for decode context parallelism; see attention.mla.graph.decode.paged.fp8.sparse.cp for the per-token lengths, the lse output, split_len_bound and the CP store. SM100 only.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[mask_str: StaticString, dcp_size: Int, dcp_interleave: Int, indices_stride: Int, physical_indices: Bool, q_absorbed: Bool = False, latent_output: Bool = False](output: T, lse: T, q: T, kv: T, input_row_offsets: T, freqs_cis: T, kv_norm_gamma: T, w_uk: T, w_uv: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, layer_idx: UInt32, scale: Float32, epsilon: Float32, scalar_args: T, sparse_indices: T, sparse_lengths: T, split_len_bound: Int32, dcp_rank: Int32, context: DeviceContext)

Was this page helpful?