IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

flare_mla_decoding_rows

def flare_mla_decoding_rows[k_t: MHAOperand, mask_t: MHAMask, dtype: DType, //, config: MHAConfig[dtype]](output: TileTensor[Engine=output.Engine, linear_idx_type=output.linear_idx_type], q: TileTensor[dtype, Engine=q.Engine, linear_idx_type=q.linear_idx_type], k: k_t, num_keys: Int, mask_functor: mask_t, scale: Float32, ctx: DeviceContext, scalar_args_buf: NullableTileTensor[.int64, Engine=scalar_args_buf.Engine, linear_idx_type=scalar_args_buf.linear_idx_type], num_partitions: Optional[Int] = None)

Runs MLA decode over a K operand with num_keys keys per entry.

The operand form of the TileTensor overload: k decides where each key row lives, so a SparseRowsMHAOperand attends a top-k selection straight from the paged cache, with no gathered copy. Each entry has one absorbed-latent KV head.

Parameters:

  • ​k_t (MHAOperand): The K operand type (inferred).
  • ​mask_t (MHAMask): The mask type (inferred).
  • ​dtype (DType): Q's element type (inferred).
  • ​config (MHAConfig[dtype]): The attention configuration.

Args:

Was this page helpful?