IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

topk_det_cluster_route

def topk_det_cluster_route[in_dtype: DType, *, ordered: Bool, deterministic: Bool](ctx: DeviceContext, N: Int, K: Int, total_seq_len: Int, bounded: Bool) -> Int

The cluster size at which persistent_topk_block_split hands these rows to the deterministic cluster top-k, or 0 where it keeps its other kernels.

The dispatch decides by calling this, so a test that asserts on it asserts on the route itself. The cluster kernel takes the rows the ranked streaming select would otherwise take under ordered=False, deterministic=True: bounded bf16 rows past the widest resident row, and only where every row gets a cluster of at least two CTAs in one wave (topk_cluster_det_cluster_size), which also keeps the rows below the SM count. At one CTA a long row is slower than the ranked select.

Parameters:

  • ​in_dtype (DType): Element type of the score buffer.
  • ​ordered (Bool): The launch's ordered contract.
  • ​deterministic (Bool): The launch's deterministic contract.

Args:

  • ​ctx (DeviceContext): Device context.
  • ​N (Int): Score columns per row (the row stride).
  • ​K (Int): Top-k count per row.
  • ​total_seq_len (Int): Number of rows.
  • ​bounded (Bool): Whether the launch passes per-row bounds.

Returns:

Int: The cluster size, or 0.

Was this page helpful?