For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

fused_qk_rms_norm_ragged_paged

def fused_qk_rms_norm_ragged_paged[dtype: DType, params: KVCacheStaticParams, page_size: Int, cache_dtype: DType, //, target: StringSlice[ImmStaticOrigin], multiply_before_cast: Bool](q_proj: TileTensor[dtype, Storage=q_proj.Storage, address_space=q_proj.address_space, linear_idx_type=q_proj.linear_idx_type], kv_collection: PagedKVCacheCollection[cache_dtype, params, page_size, scale_dtype_=kv_collection.scale_dtype_, quantization_granularity_=kv_collection.quantization_granularity_], q_gamma: TileTensor[dtype, Storage=q_gamma.Storage, address_space=q_gamma.address_space, linear_idx_type=q_gamma.linear_idx_type], k_gamma: TileTensor[dtype, Storage=k_gamma.Storage, address_space=k_gamma.address_space, linear_idx_type=k_gamma.linear_idx_type], epsilon: Float32, weight_offset: Scalar[dtype], layer_idx: UInt32, input_row_offsets: TileTensor[DType.uint32, Storage=input_row_offsets.Storage, address_space=input_row_offsets.address_space, linear_idx_type=input_row_offsets.linear_idx_type], q_output: TileTensor[dtype, Storage=q_output.Storage, address_space=q_output.address_space, linear_idx_type=q_output.linear_idx_type], context: DeviceContext)

Applies per-head RMSNorm to Q and new K-cache entries in one GPU launch.