For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
topk_det_cluster_route
def topk_det_cluster_route[in_dtype: DType, *, ordered: Bool, deterministic: Bool](ctx: DeviceContext, N: Int, K: Int, total_seq_len: Int, bounded: Bool) -> Int
The cluster size at which persistent_topk_block_split hands these rows to the deterministic cluster top-k, or 0 where it keeps its other kernels.
The dispatch decides by calling this, so a test that asserts on it
asserts on the route itself. The cluster kernel takes the rows the ranked
streaming select would otherwise take under ordered=False, deterministic=True: bounded bf16 rows past the widest resident row, and
only where every row gets a cluster of at least two CTAs in one wave
(topk_cluster_det_cluster_size), which also keeps the rows below the
SM count. At one CTA a long row is slower than the ranked select.
Parameters:
- in_dtype (
DType): Element type of the score buffer. - ordered (
Bool): The launch'sorderedcontract. - deterministic (
Bool): The launch'sdeterministiccontract.
Args:
- ctx (
DeviceContext): Device context. - N (
Int): Score columns per row (the row stride). - K (
Int): Top-k count per row. - total_seq_len (
Int): Number of rows. - bounded (
Bool): Whether the launch passes per-row bounds.
Returns:
Int: The cluster size, or 0.