IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

q_smem_shape_fused

def q_smem_shape_fused[dtype: DType, swizzle_mode: TensorMapSwizzle, *, BM: Int, group: Int, depth: Int, decoding: Bool, fuse_gqa: Bool = False, num_qk_stages: Int = Int(1)]() -> Coord[ComptimeInt[Int(1) if decoding else (BM // group)], ComptimeInt[Int(1)], ComptimeInt[max(group, Int(8)) if decoding else group], ComptimeInt[(swizzle_mode.bytes() // size_of[dtype]()) if decoding or fuse_gqa and (xor identical((BM % group), 0), True) else depth if fuse_gqa and identical(num_qk_stages, 1) else (align_up(depth, (swizzle_mode.bytes() // size_of[dtype]())) // num_qk_stages)]]

Computes the rank-4 shared-memory shape for a decoding or fused-GQA Q tensor TMA tile.

Parameters:

  • ​dtype (DType): Element type of the Q tensor.
  • ​swizzle_mode (TensorMapSwizzle): TMA swizzle mode for the Q tensor tile.
  • ​BM (Int): Tile block size in the query (row) dimension, in elements.
  • ​group (Int): Grouped-query attention group size, in query heads per KV head.
  • ​depth (Int): Head dimension of the attention layer, in elements.
  • ​decoding (Bool): Whether the kernel runs in single-token decoding mode.
  • ​fuse_gqa (Bool): Whether to fuse grouped-query attention into the tile shape (defaults to False).
  • ​num_qk_stages (Int): Number of pipeline stages used to split the Q shared-memory tile along the depth dimension (defaults to 1).

Returns:

Coord[ComptimeInt[Int(1) if decoding else (BM // group)], ComptimeInt[Int(1)], ComptimeInt[max(group, Int(8)) if decoding else group], ComptimeInt[(swizzle_mode.bytes() // size_of[dtype]()) if decoding or fuse_gqa and (xor identical((BM % group), 0), True) else depth if fuse_gqa and identical(num_qk_stages, 1) else (align_up(depth, (swizzle_mode.bytes() // size_of[dtype]())) // num_qk_stages)]]

Was this page helpful?