IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

latent_sparse_attention_ragged_paged_sm100

def latent_sparse_attention_ragged_paged_sm100[q_type: DType, out_type: DType, //, window: Int, split_k: Bool = True, portable_p_bf16: Bool = False](output: TileTensor[out_type, Engine=output.Engine, linear_idx_type=output.linear_idx_type], q: TileTensor[q_type, Engine=q.Engine, linear_idx_type=q.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type], comp_indices: TileTensor[.int32, Engine=comp_indices.Engine, linear_idx_type=comp_indices.linear_idx_type], attn_sink: TileTensor[.float32, Engine=attn_sink.Engine, linear_idx_type=attn_sink.linear_idx_type], swa_cache: PagedKVCache[scale_dtype_=swa_cache.scale_dtype_, quantization_granularity_=swa_cache.quantization_granularity_], comp_cache: PagedKVCache[scale_dtype_=comp_cache.scale_dtype_, quantization_granularity_=comp_cache.quantization_granularity_], scale: Float32, ctx: DeviceContext)

Runs latent_sparse_attention_ragged_paged on the SM100 decode when the shape and layout allow it, else on the portable GPU kernel.

Was this page helpful?