For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
mla_prefill_branch_sparse_bf16
def mla_prefill_branch_sparse_bf16[collection_t: KVCollectionT, KVInputFnType: def[width: Int](IndexList[Int(2)]) -> SIMD[.bfloat16, width] & RegisterPassable & ImplicitlyCopyable, //, indices_stride: Int, nvfp4_convert_ratio: Int = Int(0), nvfp4_convert_max_rows: Int = Int(0), AttnSinkPtrType: OptionalPointer = NullPointer[.float32], *, kernels: MLAGraphKernels](output: TileTensor[.bfloat16, Engine=output.Engine, linear_idx_type=output.linear_idx_type], q: TileTensor[.bfloat16, Engine=q.Engine, linear_idx_type=q.linear_idx_type], input_row_offsets: TileTensor[.uint32, Engine=input_row_offsets.Engine, linear_idx_type=input_row_offsets.linear_idx_type], freqs_cis: TileTensor[Engine=freqs_cis.Engine, linear_idx_type=freqs_cis.linear_idx_type], kv_norm_gamma: TileTensor[Engine=kv_norm_gamma.Engine, linear_idx_type=kv_norm_gamma.linear_idx_type], kv_collection: collection_t, layer_idx: UInt32, scale: Float32, epsilon: Float32, w_uk: TileTensor[.bfloat16, Engine=w_uk.Engine, linear_idx_type=w_uk.linear_idx_type], w_uv: TileTensor[.bfloat16, Engine=w_uv.Engine, linear_idx_type=w_uv.linear_idx_type], kv_input_fn: KVInputFnType, ctx: DeviceContext, d_indices: MutPointer[Int32, MutAnyOrigin], topk_lengths: MutPointer[Int32, MutAnyOrigin], attn_sink_ptr: AttnSinkPtrType = null_pointer[AttnSinkPtrType]())
Sparse MLA prefill branch (DSv3.2/GLM absorbed shape, BF16 weights).
BF16 analogue of mla_prefill_branch_sparse_fp8: reuses
mla_decode_branch_bf16's absorbed-Q construction (q_nope up-proj via w_uk
- RoPE/RMSNorm) and the identical
w_uvoutput up-projection, and swaps the attention call to the existingmla_sm100_prefill_sparsekernel over the paged BF16 latent cache. The caller (.sparseop) has already remappedd_indicesfrom logical to physical rows, so they are passed straight through. Only supported for a BF16 KV cache.