For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo trait
IndexerScoreHook
Reads or rewrites indexer scores in the scorer's epilogue.
The scorer hands every score it produces to score, as the value it is
about to store, so a hook sees exactly what the top-k later reads and adds
no pass over the score matrix. The lanes of a warp call score together
for one score row, lane l holding key lane_keys * j + l for some j,
so a hook may combine aligned blocks of up to lane_keys keys with warp
shuffles.
Implemented traits
AnyType,
Copyable,
Deinitable,
DevicePassable,
ImplicitlyCopyable,
Movable,
RegisterPassable,
TrivialRegisterPassable
comptime members
enabled
comptime enabled
Whether the hook runs; False compiles every call away.
Required methods
window
def window(self, ctx: DeviceContext, first_row: Int, rows: Int) -> Self
Prepares a window of score rows and returns the hook to run on it.
Args:
- ctx (
DeviceContext): Device context; work enqueued here precedes the scorer. - first_row (
Int): Query token of the window's row 0. - rows (
Int): Rows in the window.
Returns:
_Self: The hook whose row 0 is query token first_row.
window_rows
def window_rows(self) -> Int
Returns the most score rows one window may hold.
Returns:
Int: The row cap; a hook with per-window scratch sizes it by this.
after_window
def after_window(self, ctx: DeviceContext, first_row: Int, rows: Int)
Runs after a window of score rows is scored.
Args:
score
def score[dtype: DType, lane_keys: Int](self, value: Scalar[dtype], row: Int32, key: Int32, key_live: Bool, row_live: Bool) -> Scalar[dtype]
Returns the score to store for (row, key).
Parameters:
- dtype (
DType): Score element type. - lane_keys (
Int): Consecutive keys a warp holds, one per lane, starting at a multiple oflane_keys.
Args:
- value (
Scalar[dtype]): The score, rounded todtype. - row (
Int32): Window row of the query token. - key (
Int32): Key column. - key_live (
Bool): Whether the key is below the row's bound; only such scores are stored. - row_live (
Bool): Whether the row is in the window; the same on every lane. A dead row must not be read or written.
Returns:
Scalar[dtype]: The score to store.