For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
MLACPMergeTopk
struct MLACPMergeTopk
Registers the attention.mla.cp_merge_topk graph op.
Merges a decode context-parallel group's local indexer selections (see
attention.mla.indexer.ragged.float8.paged.remap.cp) into the group's top-k and
keeps this rank's share, compacted to the front of each row. With
halves = 2 (an NVFP4 MLA cache, see
attention.mla.indexer.ragged.float8.paged.remap.cp.nvfp4) each row is the
selections' value rows then their scale-pool rows, and both halves are
compacted the same way.
Implemented traits
Methods
execute
static def execute[*, dcp_size: Int, k: Int, halves: Int = Int(1)](rows: T, lengths: T, local_rows: T, scores: T, dcp_rank: T, ctx: DeviceContext)
Keeps this rank's selections that are in the group's top-k.
Parameters:
- dcp_size (
Int): Ranks in the decode context-parallel group. - k (
Int): Selections per token, on each rank and after the merge. - halves (
Int): Rows per selection: 1, or 2 for a value row and its scale-pool rowkcolumns later (defaults to 1).
Args:
- rows (
T):[T, halves * k]int32 rows of this rank's selections in the group's top-k, compacted to the front, then-1; withhalves = 2their scale rows follow at columnk. - lengths (
T):[T]int32 count ofrows' selected slots (per half). - local_rows (
T):[T, halves * k]int32 this rank's local selections. - scores (
T):[T, dcp_size * k]bf16 every rank's selection scores, rank-major,-inffor padding. - dcp_rank (
T): This rank in[0, dcp_size), int32[1]on the device. - ctx (
DeviceContext): Device context for GPU execution.