IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_kv_cache_pcp_pack_rows_paged_scaled

struct Struct_kv_cache_pcp_pack_rows_paged_scaled

Registers the kv_cache.pcp_pack_rows.paged.scaled graph op: the kv_cache.pcp_pack_rows.paged op over a cache with a scale pool, whose operands arrive in the order flatten_without_attention_dispatch_metadata emits for quantized cache params.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[col_offset: Int, dcp_size: Int, dcp_interleave: Int](output: T, row_offsets: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, kv_scales: T, scales_page_stride: T, scales_lookup_table: T, layer_idx: UInt32, dcp_rank: Int32, context: DeviceContext)

Was this page helpful?