IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

TopKPerRow

struct TopKPerRow

Registers the mo.top_k.per_row graph op with the graph compiler.

mo.top_k over the last axis of a [rows, n] input, with a count per row: row r keeps the first k[r] picks mo.top_k with max_k makes for it, in descending order, and pads the rest of its max_k slots with the dead value and index -1. Equal values may come out in any order, and which of them fill the last slots is unspecified. The input must not contain NaN. On GPU, float32 and bfloat16 rows with max_k <= 2048 run the block-wide top-k in topk_per_row, which selects all picks without one serial reduction per pick.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[dtype: DType, //, max_k: Int, target: StringSpan[ImmStaticOrigin]](values: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=values.static_spec], indices: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=indices.static_spec], input: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=input.static_spec], k: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=k.static_spec], ctx: DeviceContextBase)

Executes the mo.top_k.per_row graph op.

Parameters:

  • ​dtype (DType): Element type of the input and values.
  • ​max_k (Int): Output width, the largest count any row may ask for.
  • ​target (StringSpan[ImmStaticOrigin]): Compilation target string.

Args:

Raises:

Error: If the operation parameters are invalid.

Was this page helpful?