IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

mla_sparse_attention_flat

def mla_sparse_attention_flat[cache_t: KVCacheT, //, num_heads: Int, depth: Int, fp8_scale_bf16: Bool = False, scale_block_size_override: Int = Int(0)](output: TileTensor[Engine=output.Engine, linear_idx_type=output.linear_idx_type], q: TileTensor[Engine=q.Engine, linear_idx_type=q.linear_idx_type], cache: cache_t, indices: MutPointer[Int32, MutAnyOrigin], indices_stride: Int, topk_lengths: TileTensor[.uint32, Engine=topk_lengths.Engine, linear_idx_type=topk_lengths.linear_idx_type], attn_sink_ptr: Optional[Pointer[Float32, ImmutAnyOrigin]], scale: Float32, ctx: DeviceContext, scales_ptr_override: Optional[Pointer[Float32, MutAnyOrigin]] = None)

Runs independent physical sparse rows with the shared attention core.

All queries form one scheduling segment. Their per-query selected lengths are authoritative; cache batch lengths do not restrict these physical rows. The form is null-masked, so a row's -1 slots are excluded by their index wherever they sit rather than by position.

Was this page helpful?