IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

k_rope_v_store_kernel

def k_rope_v_store_kernel[freq_dtype: DType](k: TileTensor[.bfloat16, Engine=k.Engine, address_space=k.address_space, linear_idx_type=k.linear_idx_type], v: TileTensor[.bfloat16, Engine=v.Engine, address_space=v.address_space, linear_idx_type=v.linear_idx_type], row_offsets: TileTensor[.uint32, Engine=row_offsets.Engine, address_space=row_offsets.address_space, linear_idx_type=row_offsets.linear_idx_type], freqs: TileTensor[freq_dtype, Engine=freqs.Engine, address_space=freqs.address_space, linear_idx_type=freqs.linear_idx_type], k_cache: PagedKVCache[scale_dtype_=k_cache.scale_dtype_, quantization_granularity_=k_cache.quantization_granularity_], v_cache: PagedKVCache[k_cache.dtype_, k_cache.kv_params_, k_cache.page_size, k_cache.blocks_origin, k_cache.blocks_engine, k_cache.cache_lengths_origin, k_cache.cache_lengths_engine_, k_cache.lookup_table_origin, k_cache.lookup_table_engine, k_cache.scales_origin, k_cache.scales_engine, scale_dtype_=k_cache.scale_dtype_, quantization_granularity_=k_cache.quantization_granularity_], value_scale: Float32)

Rotates K and writes scaled V, for a call that ropes no query.

Constraints:

As rope_qkv_store_kernel, minus the query: the cache must be unquantized, non-MLA, BF16, and not split.

Parameters:

  • ​freq_dtype (DType): Element type of the RoPE coefficients (inferred).

Args:

Was this page helpful?