IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

topk_per_row_gpu

def topk_per_row_gpu[dtype: DType, //](ctx: DeviceContext, scores: ImmPointer[Scalar[dtype], ImmutAnyOrigin], k: ImmPointer[Int64, ImmutAnyOrigin], out_vals: MutPointer[Scalar[dtype], MutAnyOrigin], out_idxs: MutPointer[Int64, MutAnyOrigin], rows: Int, n: Int, max_k: Int)

Selects the first k[r] picks of each row's top-max_k.

Row r gets the k[r] largest scores in descending order as (value, column) pairs, with equal scores in any order. Every other slot of its max_k holds (-inf, -1).

Parameters:

  • ​dtype (DType): Score and value dtype, float32 or bfloat16.

Args:

Was this page helpful?