For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
q_smem_shape_fused
def q_smem_shape_fused[dtype: DType, swizzle_mode: TensorMapSwizzle, *, BM: Int, group: Int, depth: Int, decoding: Bool, fuse_gqa: Bool = False, num_qk_stages: Int = Int(1)]() -> Coord[ComptimeInt[Int(1) if decoding else (BM // group)], ComptimeInt[Int(1)], ComptimeInt[max(group, Int(8)) if decoding else group], ComptimeInt[(swizzle_mode.bytes() // size_of[dtype]()) if decoding or fuse_gqa and (xor identical((BM % group), 0), True) else depth if fuse_gqa and identical(num_qk_stages, 1) else (align_up(depth, (swizzle_mode.bytes() // size_of[dtype]())) // num_qk_stages)]]
Computes the rank-4 shared-memory shape for a decoding or fused-GQA Q tensor TMA tile.
Parameters:
- dtype (
DType): Element type of the Q tensor. - swizzle_mode (
TensorMapSwizzle): TMA swizzle mode for the Q tensor tile. - BM (
Int): Tile block size in the query (row) dimension, in elements. - group (
Int): Grouped-query attention group size, in query heads per KV head. - depth (
Int): Head dimension of the attention layer, in elements. - decoding (
Bool): Whether the kernel runs in single-token decoding mode. - fuse_gqa (
Bool): Whether to fuse grouped-query attention into the tile shape (defaults toFalse). - num_qk_stages (
Int): Number of pipeline stages used to split the Q shared-memory tile along the depth dimension (defaults to 1).
Returns: