IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

matmul_sm100_blockwise_scaled_fp8

def matmul_sm100_blockwise_scaled_fp8[*, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], ctx: DeviceContext)

Enqueues a blockwise scaled FP8 GEMM on SM100 with 1D A-scales and 2D B-scales onto the supplied device context.

Validates the operand dtypes, ranks, and scale granularities, builds the TMA tile descriptors for A, B, and A-scales, and launches the matmul_sm100_blockwise_scaled_fp8_1d2d_wrapper kernel over a two-dimensional grid covering the M and N dimensions.

Parameters:

  • ​transpose_b (Bool): Whether B is loaded transposed; must be True (only the transposed-B path is supported).
  • ​umma_shape (Tuple[Int, Int, Int]): 3D MMA instruction shape (MMA_M, MMA_N, MMA_K) used by the TCgen05 tensor core operations.
  • ​block_tile_shape (Tuple[Int, Int, Int]): 3D block tile shape (BM, BN, BK) giving the per-CTA tile dimensions along the M, N, and K axes. BK must be 64 or 128.
  • ​a_swizzle (TensorMapSwizzle): TMA swizzle mode applied to A shared memory loads (defaults to TensorMapSwizzle.SWIZZLE_128B).
  • ​b_swizzle (TensorMapSwizzle): TMA swizzle mode applied to B shared memory loads (defaults to TensorMapSwizzle.SWIZZLE_128B).

Args:

def matmul_sm100_blockwise_scaled_fp8[EpilogueFnType: ElementwiseEpilogueFn, //, *, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], epilogue_fn: EpilogueFnType, ctx: DeviceContext)

Enqueues a blockwise scaled FP8 GEMM on SM100 (see the other overload), delivering each output value to an epilogue closure instead of writing c.

Parameters:

Args:

def matmul_sm100_blockwise_scaled_fp8[ComputeFnType: ElementwiseComputeFn, //, *, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], compute_fn: ComputeFnType, ctx: DeviceContext)

Enqueues a blockwise scaled FP8 GEMM on SM100 (see the other overload), storing compute_fn(idx, value) into c.

Parameters:

Args:

Was this page helpful?