For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
BlockSplitTopk
struct BlockSplitTopk[deterministic: Bool = False]
persistent_topk_block_split, the indexers' default top-k.
deterministic=True keeps a row's column order unspecified but makes it
reproducible: one input always gives the same columns in the same slots.
A consumer that reduces over the slots in order needs that, the sparse MLA
softmax among them; without it the slots are filled in atomic arrival
order, so that consumer's fp32 sum order, and the low bits of its output,
change from launch to launch. The price is that few-row, long-row shapes
give up the cluster-cooperative kernel, which only the default contract
allows.
Parameters
- deterministic (
Bool): Whether one input must always give one output, passed topersistent_topk_block_split.
Implemented traits
AnyType,
Deinitable,
IndexerTopk,
Movable
Methods
select
static def select[in_dtype: DType, RB: RowBound](ctx: DeviceContext, scores: ImmPointer[Scalar[in_dtype], ImmutAnyOrigin], out_idxs: MutPointer[Int32, MutAnyOrigin], n: Int, k: Int, rows: Int, row_bounds: RB)
Writes the top-k columns of each row of scores to out_idxs.
Parameters:
Args:
- ctx (
DeviceContext): Device context. - scores (
ImmPointer[Scalar[in_dtype], ImmutAnyOrigin]): Row-major scores,[rows, n]. - out_idxs (
MutPointer[Int32, MutAnyOrigin]): Row-major output,[rows, k]. - n (
Int): Score columns per row, the row stride. - k (
Int): Columns selected per row. - rows (
Int): Score rows. - row_bounds (
RB): Per-row live column counts and output sinks.