IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

fp8_index_score_amd

def fp8_index_score_amd[dtype: DType, KOperand: MHAOperand, KSOperand: MHAOperand, num_heads: Int, depth: Int, _is_cache_length_accurate: Bool, causal: Bool = False, kpool: Int = Int(1), pool_row_tokens: Int = Int(1)](output: TileTensor[.float32, Engine=output.Engine, address_space=output.address_space, linear_idx_type=output.linear_idx_type], q: TileTensor[dtype, Engine=q.Engine, address_space=q.address_space, linear_idx_type=q.linear_idx_type], q_s: TileTensor[.float32, Engine=q_s.Engine, address_space=q_s.address_space, linear_idx_type=q_s.linear_idx_type], k_operand: KOperand, ks_operand: KSOperand, valid_length: TileTensor[.uint32, Engine=valid_length.Engine, address_space=valid_length.address_space, linear_idx_type=valid_length.linear_idx_type], batch_size: Int, max_seq_len: Int, max_num_keys: Int, ctx: DeviceContextBase)

Enqueues the gfx950 MFMA FP8 indexer scorer (_fp8_index_score_amd_kernel).

AMD counterpart of fp8_index_score_sm100: writes the same [total_seq, max_num_keys] score buffer as the scalar fp8_index_kernel, with OOB keys left untouched (the caller's -inf fill or bounded top-k covers them).

Parameters:

  • ​dtype (DType): FP8 element type of Q and K (float8_e4m3fn).
  • ​KOperand (MHAOperand): MHAOperand type for the K values.
  • ​KSOperand (MHAOperand): MHAOperand type for the per-token K scales.
  • ​num_heads (Int): Query index heads (multiple of 32).
  • ​depth (Int): Head dimension (multiple of 64).
  • ​_is_cache_length_accurate (Bool): When False, num_keys = cache_length + seq_len; when True, cache_length already includes the new tokens.
  • ​causal (Bool): Fuse the causal bound into the store guard (token t sees keys up to cache_len + t + 1); forbidden slots are never written.
  • ​kpool (Int): Tokens per scored key; see _fp8_index_score_amd_kernel.
  • ​pool_row_tokens (Int): Cache rows a pooled key spans; see _fp8_index_score_amd_kernel.

Args:

Was this page helpful?