IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

indexer_k_norm_rope_store

def indexer_k_norm_rope_store[fp4: Bool, ratio: Int](keys: TileTensor[.bfloat16, Engine=keys.Engine, address_space=keys.address_space, linear_idx_type=keys.linear_idx_type], positions: TileTensor[.int64, Engine=positions.Engine, address_space=positions.address_space, linear_idx_type=positions.linear_idx_type], frequencies: TileTensor[.float32, Engine=frequencies.Engine, address_space=frequencies.address_space, linear_idx_type=frequencies.linear_idx_type], weight: TileTensor[.bfloat16, Engine=weight.Engine, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], slots: TileTensor[.int64, Engine=slots.Engine, address_space=slots.address_space, linear_idx_type=slots.linear_idx_type], cache: TileTensor[.uint8, Engine=cache.Engine, address_space=cache.address_space, linear_idx_type=cache.linear_idx_type], page_size: Int, epsilon: Float32, ctx: DeviceContext)

Store 128-channel index keys after RMSNorm and 64-channel GPT-J RoPE.

Constraints:

The FP4 path requires SM100 or newer. Each page contains at least page_size records of MXFP4_INDEX_RECORD_BYTES for FP4, or of FP8_INDEX_RECORD_BYTES for FP8. Only group-boundary rows with nonnegative slots write.

Parameters:

  • ​fp4 (Bool): Select MXFP4 E2M1/UE8M0; otherwise use E4M3/FP32.
  • ​ratio (Int): Compression group size, either 1 or 2.

Args:

Raises:

If a row of keys, frequencies, or cache is not contiguous and aligned.

Was this page helpful?