IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

nvfp4_kv_store_row

def nvfp4_kv_store_row[cache_t: KVCacheT, dtype: DType, //, head_dim: Int, simd_width: Int](active: Bool, idx: Int, batch_idx: Int, post_seq_idx: Int, head_idx: Int, cache: cache_t, row: SIMD[dtype, simd_width])

Stores one (token, head) row held simd_width columns per thread.

A uint8 cache receives the row as NVFP4 (see nvfp4_kv_write_row); any other cache receives it cast to its dtype. idx is the first column the thread holds, and the threads of a scale group must be adjacent lanes of one warp, because the group amax is exchanged with shuffle_xor. Every lane of the warp must call this, active only masks the stores.

Was this page helpful?