For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
hisparse_resolve
def hisparse_resolve[page_size: Int, hot_pages: Int, num_groups: Int, max_misses: Int, hash_capacity: Int](physical: MutPointer[Int32], logical: ImmPointer[Int32], row_offsets: ImmPointer[UInt32], lookup_table: ImmPointer[UInt32], host_table: ImmPointer[Int32], hot_table: ImmPointer[UInt32], cache: MutPointer[UInt8], page_stride_bytes: Int, page_data_bytes: Int, num_batches: Int, lut_cols: Int, top_k: Int, group: Int, ctx: DeviceContext)
Rewrites selections on host-only pages to hot-buffer rows.
Parameters:
- page_size (
Int): Rows per cache page. - hot_pages (
Int): Pages in each request's hot buffer. - num_groups (
Int): Indexer layers that keep their own LRU state. - max_misses (
Int): Most distinct host-only selections one request may make per call; covers every row's top-k under speculation. - hash_capacity (
Int): Entries in the per-call selection hash, a power of two abovemax_misses.
Args:
- physical (
MutPointer[Int32]):[tokens, top_k]physical rows from the indexer, rewritten in place. - logical (
ImmPointer[Int32]):[tokens, top_k]logical positions of the same selections. - row_offsets (
ImmPointer[UInt32]):[num_batches + 1]ragged token offsets. - lookup_table (
ImmPointer[UInt32]):[num_batches, lut_cols]cache pages; 0 marks a host-only page. - host_table (
ImmPointer[Int32]):[num_batches, lut_cols]host mirror pages,-1where a page has no copy. - hot_table (
ImmPointer[UInt32]):[num_batches, hot_pages + 1]hot pages and, last, the state page; a zero state page means no hot region. - cache (
MutPointer[UInt8]): The cache's bytes; pages arepage_stride_bytesapart. - page_stride_bytes (
Int): Bytes from one cache page to the next. - page_data_bytes (
Int): Bytes of data in a page, which bound the state. - num_batches (
Int): Requests in the batch. - lut_cols (
Int): Columns of the lookup and host tables. - top_k (
Int): Selections per token. - group (
Int): This indexer layer's LRU state index. - ctx (
DeviceContext): Device context to enqueue on.