IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_kv_cache_pcp_pack_rows_paged

struct Struct_kv_cache_pcp_pack_rows_paged

Registers the kv_cache.pcp_pack_rows.paged graph op.

Packs the cached rows one context-parallel rank owns, as raw bytes, into a column window of output, for an all-gather over the group. See nn.kv_cache_gather.kv_cache_pack_owned_rows. The .scaled variant also packs each row's scales.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[col_offset: Int, dcp_size: Int, dcp_interleave: Int](output: T, row_offsets: T, kv_blocks: T, page_stride: T, cache_lengths: T, kv_lookup_table: T, max_prompt_length: T, max_cache_length: T, layer_idx: UInt32, dcp_rank: Int32, context: DeviceContext)

Was this page helpful?