IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_rope_qkv_store_ragged_paged

struct Struct_rope_qkv_store_ragged_paged

Registers head-major Q/K rotation and paged K/V stores.

The drafter's counterpart to kv_cache.chunked_qkv_rope_store.ragged.paged: it takes the three projections separately, so a layer whose Q, K and V never share a buffer writes its cache in one launch instead of storing K raw, storing V, and then rotating K in place.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute(output: T, q: T, k: T, v: T, input_row_offsets: T, freqs_cis: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, layer_idx: UInt32, value_scale: Float32, ctx: DeviceContext)

Was this page helpful?