For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
IndexSmemStorage
struct IndexSmemStorage[dtype: DType, num_heads: Int, depth: Int, BN: Int]
Holds shared-memory buffers for the query, key, and scratch tiles used by the FP8 index kernel.
Parameters
- dtype (
DType): Data type of the query and key tiles. - num_heads (
Int): Number of attention heads stored in the query tile. - depth (
Int): Per-head feature depth of the query and key tiles. - BN (
Int): Number of key rows staged in shared memory per block tile.
Fields
- q_smem (
Array[Scalar[dtype], (num_heads * depth)]): - k_smem (
Array[Scalar[dtype], (BN * depth)]): - scratch (
Array[Float32, (BN * Int(8))]):
Implemented traits
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!