For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
hisparse_gather_group
def hisparse_gather_group[page_size: Int, hot_pages: Int, num_groups: Int, max_misses: Int, hash_capacity: Int](cache: MutPointer[UInt8], host: ImmPointer[UInt8], hot_table: ImmPointer[UInt32], page_stride_bytes: Int, host_page_bytes: Int, first_layer_offset_bytes: Int, row_bytes: Int, layer_count: Int, layer_stride_bytes: Int, num_batches: Int, ctx: DeviceContext)
Copies the last resolve's misses for consecutive layers in one launch.
Parameters:
- page_size (
Int): Rows per cache page. - hot_pages (
Int): Pages in each request's hot buffer. - num_groups (
Int): Indexer layers that keep their own LRU state. - max_misses (
Int): As passed tohisparse_resolve. - hash_capacity (
Int): As passed tohisparse_resolve.
Args:
- cache (
MutPointer[UInt8]): The cache's bytes; pages arepage_stride_bytesapart. - host (
ImmPointer[UInt8]): The host mirror's bytes, readable by the device; pages arehost_page_bytesapart and laid out like cache pages. - hot_table (
ImmPointer[UInt32]):[num_batches, hot_pages + 1], as passed to the resolve. - page_stride_bytes (
Int): Bytes from one cache page to the next. - host_page_bytes (
Int): Bytes from one host page to the next. - first_layer_offset_bytes (
Int): Byte offset of the first layer's rows within a page. - row_bytes (
Int): Bytes per row, a multiple of 16. - layer_count (
Int): How many consecutive layers to copy. - layer_stride_bytes (
Int): Bytes from one layer's rows to the next within a page, a multiple of 16. - num_batches (
Int): Requests in the batch. - ctx (
DeviceContext): Device context to enqueue on.