IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

SparseRowLayout

struct SparseRowLayout

Describes how a sparse index producer packs one token's row.

Every producer packs its valid entries first and pads with -1, so the number of valid entries is a closed-form function of how many keys the token sees. The decode mask uses it to keep -1 slots out of the softmax without reading the indices.

A k-pool producer (kpool_expand_topk_kernel) selects up to pool_topk pools of kpool keys, then, with select_tail, the visible % kpool keys no complete pool covers yet. kpool = 1 is a plain top-k row; pool_topk = 0 leaves its length unbounded, since the kernel's slot count already caps it.

Fields​

  • ​kpool (Int): Keys per selected pool; 1 for plain top-k.
  • ​pool_topk (Int): Maximum number of selected pools; 0 for no bound.
  • ​select_tail (Bool): Whether the row appends the incomplete trailing pool.

Implemented traits​

AnyType, Copyable, Deinitable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

Methods​

__init__​

def __init__() -> Self

Creates the plain top-k layout, which adds no bound.

is_bounded​

def is_bounded(self) -> Bool

Returns whether valid_len can be shorter than its argument.

Returns:

Bool: True when the layout bounds a row below the token's key count.

valid_len​

def valid_len(self, visible: Int) -> Int

Returns the number of valid entries in a row.

Args:

  • ​visible (Int): Number of keys visible to the token the row was computed for.

Returns:

Int: The row's valid length; slots at or past it hold -1.

row_limit​

def row_limit(self, visible: Int, selection_lag: Int) -> Int

Returns the valid length of a row read selection_lag tokens late.

Args:

  • ​visible (Int): Number of keys visible to the reading token.
  • ​selection_lag (Int): Positions between the token the row was computed for and the reading token.

Returns:

Int: The row's valid length, or Int.MAX when nothing bounds it.

Was this page helpful?