IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

q_smem_shape_prefill

def q_smem_shape_prefill[dtype: DType, swizzle_mode: TensorMapSwizzle, *, BM: Int, depth: Int, num_qk_stages: Int = Int(1)]() -> Coord[ComptimeInt[BM], ComptimeInt[Int(1)], ComptimeInt[depth if (num_qk_stages == Int(1)) else (align_up(depth, (swizzle_mode.bytes() // size_of[dtype]())) // num_qk_stages)]]

Computes the rank-3 shared-memory shape for a prefill (non-decoding, non-fused) Q tensor TMA tile.

Parameters:

  • ​dtype (DType): Element type of the Q tensor.
  • ​swizzle_mode (TensorMapSwizzle): TMA swizzle mode for the Q tensor tile.
  • ​BM (Int): Tile block size in the query (row) dimension, in elements.
  • ​depth (Int): Head dimension of the attention layer, in elements.
  • ​num_qk_stages (Int): Number of pipeline stages used to split the Q shared-memory tile along the depth dimension (defaults to 1).

Returns:

Coord[ComptimeInt[BM], ComptimeInt[Int(1)], ComptimeInt[depth if (num_qk_stages == Int(1)) else (align_up(depth, (swizzle_mode.bytes() // size_of[dtype]())) // num_qk_stages)]]

Was this page helpful?