For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
SparseRowLayout
struct SparseRowLayout
Describes how a sparse index producer packs one token's row.
Every producer packs its valid entries first and pads with -1, so the
number of valid entries is a closed-form function of how many keys the
token sees. The decode mask uses it to keep -1 slots out of the
softmax without reading the indices.
A k-pool producer (kpool_expand_topk_kernel) selects up to
pool_topk pools of kpool keys, then, with select_tail, the
visible % kpool keys no complete pool covers yet. kpool = 1 is a
plain top-k row; pool_topk = 0 leaves its length unbounded, since the
kernel's slot count already caps it.
Fields
- kpool (
Int): Keys per selected pool; 1 for plain top-k. - pool_topk (
Int): Maximum number of selected pools; 0 for no bound. - select_tail (
Bool): Whether the row appends the incomplete trailing pool.
Implemented traits
AnyType,
Copyable,
Deinitable,
ImplicitlyCopyable,
Movable,
RegisterPassable,
TrivialRegisterPassable
Methods
__init__
def __init__() -> Self
Creates the plain top-k layout, which adds no bound.
is_bounded
def is_bounded(self) -> Bool
Returns whether valid_len can be shorter than its argument.
Returns:
Bool: True when the layout bounds a row below the token's key count.
valid_len
def valid_len(self, visible: Int) -> Int
Returns the number of valid entries in a row.
Args:
- visible (
Int): Number of keys visible to the token the row was computed for.
Returns:
Int: The row's valid length; slots at or past it hold -1.
row_limit
def row_limit(self, visible: Int, selection_lag: Int) -> Int
Returns the valid length of a row read selection_lag tokens late.
Args:
- visible (
Int): Number of keys visible to the reading token. - selection_lag (
Int): Positions between the token the row was computed for and the reading token.
Returns:
Int: The row's valid length, or Int.MAX when nothing bounds it.