IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

QkArgs

struct QkArgs[cache_dtype: DType, freq_dtype: DType = .float32]

qk_norm_rope_store's operands, one kernel parameter.

Fields​

  • ​qkv (In[.bfloat16]): The projection, qkv_stride elements a token.
  • ​q_out (Out[.bfloat16]): The roped Q [tokens, n_q * HEAD_DIM].
  • ​freqs (In[freq_dtype]): Interleaved (cos, sin) pairs [positions, HEAD_DIM].
  • ​offsets (In[.uint32]): Ragged row offsets [batch + 1].
  • ​cache_lengths (In[.uint32]):
  • ​lut (In[.uint32]): The page table, lut_cols pages a sequence.
  • ​k_rows (Out[cache_dtype]): This layer's key plane: rows of n_kv * HEAD_DIM, page_rows rows from one page to the next.
  • ​v_rows (Out[cache_dtype]):
  • ​qkv_stride (Int64):
  • ​tokens (Int64):
  • ​batch (Int64):
  • ​lut_cols (Int64):
  • ​page_rows (Int64):
  • ​eps (Float32):
  • ​q_scale (Float32): The Q factor, already rounded to bf16.

Implemented traits​

AnyType, Copyable, Deinitable, DevicePassable, ImplicitlyCopyable, Movable

comptime members​

device_type​

comptime device_type = QkArgs[cache_dtype, freq_dtype]

Methods​

get_type_name​

static def get_type_name() -> String

Returns:

String

Was this page helpful?