IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MLACPMergeTopk

struct MLACPMergeTopk

Registers the attention.mla.cp_merge_topk graph op.

Merges a decode context-parallel group's local indexer selections (see attention.mla.indexer.ragged.float8.paged.remap.cp) into the group's top-k and keeps this rank's share, compacted to the front of each row. With halves = 2 (an NVFP4 MLA cache, see attention.mla.indexer.ragged.float8.paged.remap.cp.nvfp4) each row is the selections' value rows then their scale-pool rows, and both halves are compacted the same way.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[*, dcp_size: Int, k: Int, halves: Int = Int(1)](rows: T, lengths: T, local_rows: T, scores: T, dcp_rank: T, ctx: DeviceContext)

Keeps this rank's selections that are in the group's top-k.

Parameters:

  • ​dcp_size (Int): Ranks in the decode context-parallel group.
  • ​k (Int): Selections per token, on each rank and after the merge.
  • ​halves (Int): Rows per selection: 1, or 2 for a value row and its scale-pool row k columns later (defaults to 1).

Args:

  • ​rows (T): [T, halves * k] int32 rows of this rank's selections in the group's top-k, compacted to the front, then -1; with halves = 2 their scale rows follow at column k.
  • ​lengths (T): [T] int32 count of rows' selected slots (per half).
  • ​local_rows (T): [T, halves * k] int32 this rank's local selections.
  • ​scores (T): [T, dcp_size * k] bf16 every rank's selection scores, rank-major, -inf for padding.
  • ​dcp_rank (T): This rank in [0, dcp_size), int32 [1] on the device.
  • ​ctx (DeviceContext): Device context for GPU execution.

Was this page helpful?