IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

BlockSplitTopk

struct BlockSplitTopk[deterministic: Bool = False]

persistent_topk_block_split, the indexers' default top-k.

deterministic=True keeps a row's column order unspecified but makes it reproducible: one input always gives the same columns in the same slots. A consumer that reduces over the slots in order needs that, the sparse MLA softmax among them; without it the slots are filled in atomic arrival order, so that consumer's fp32 sum order, and the low bits of its output, change from launch to launch. The price is that few-row, long-row shapes give up the cluster-cooperative kernel, which only the default contract allows.

Parameters​

  • ​deterministic (Bool): Whether one input must always give one output, passed to persistent_topk_block_split.

Implemented traits​

AnyType, Deinitable, IndexerTopk, Movable

Methods​

select​

static def select[in_dtype: DType, RB: RowBound](ctx: DeviceContext, scores: ImmPointer[Scalar[in_dtype], ImmutAnyOrigin], out_idxs: MutPointer[Int32, MutAnyOrigin], n: Int, k: Int, rows: Int, row_bounds: RB)

Writes the top-k columns of each row of scores to out_idxs.

Parameters:

  • ​in_dtype (DType): Score element type.
  • ​RB (RowBound): Type of row_bounds.

Args:

Was this page helpful?