For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
matmul_sm100_blockwise_scaled_fp8
def matmul_sm100_blockwise_scaled_fp8[*, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], ctx: DeviceContext)
Enqueues a blockwise scaled FP8 GEMM on SM100 with 1D A-scales and 2D B-scales onto the supplied device context.
Validates the operand dtypes, ranks, and scale granularities, builds the
TMA tile descriptors for A, B, and A-scales, and launches the
matmul_sm100_blockwise_scaled_fp8_1d2d_wrapper kernel over a
two-dimensional grid covering the M and N dimensions.
Parameters:
- transpose_b (
Bool): Whether B is loaded transposed; must beTrue(only the transposed-B path is supported). - umma_shape (
Tuple[Int, Int, Int]): 3D MMA instruction shape(MMA_M, MMA_N, MMA_K)used by the TCgen05 tensor core operations. - block_tile_shape (
Tuple[Int, Int, Int]): 3D block tile shape(BM, BN, BK)giving the per-CTA tile dimensions along the M, N, and K axes.BKmust be 64 or 128. - a_swizzle (
TensorMapSwizzle): TMA swizzle mode applied to A shared memory loads (defaults toTensorMapSwizzle.SWIZZLE_128B). - b_swizzle (
TensorMapSwizzle): TMA swizzle mode applied to B shared memory loads (defaults toTensorMapSwizzle.SWIZZLE_128B).
Args:
- c (
TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Rank-2 outputTileTensoraccumulating the scaled GEMM result. - a (
TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Rank-2TileTensorof A operands withfloat8_e4m3fnelements. - b (
TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Rank-2TileTensorof B operands withfloat8_e4m3fnelements. - a_scales (
TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type]): Rank-2TileTensorof A scale factors withfloat32elements and 1D scaling granularity along K. - b_scales (
TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type]): Rank-2TileTensorof B scale factors withfloat32elements and 2D scaling granularity along N and K. - ctx (
DeviceContext): Device context used to enqueue the kernel launch.
def matmul_sm100_blockwise_scaled_fp8[EpilogueFnType: ElementwiseEpilogueFn, //, *, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], epilogue_fn: EpilogueFnType, ctx: DeviceContext)
Enqueues a blockwise scaled FP8 GEMM on SM100 (see the other overload), delivering each output value to an epilogue closure instead of writing c.
Parameters:
- EpilogueFnType (
ElementwiseEpilogueFn): Type ofepilogue_fn. - transpose_b (
Bool): Whether B is loaded transposed; must beTrue. - umma_shape (
Tuple[Int, Int, Int]): 3D MMA instruction shape(MMA_M, MMA_N, MMA_K). - block_tile_shape (
Tuple[Int, Int, Int]): 3D block tile shape(BM, BN, BK). - a_swizzle (
TensorMapSwizzle): TMA swizzle mode applied to A shared memory loads. - b_swizzle (
TensorMapSwizzle): TMA swizzle mode applied to B shared memory loads.
Args:
- c (
TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Rank-2 outputTileTensor; only its shape is read. - a (
TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Rank-2TileTensorof A operands withfloat8_e4m3fnelements. - b (
TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Rank-2TileTensorof B operands withfloat8_e4m3fnelements. - a_scales (
TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type]): Rank-2TileTensorof A scale factors. - b_scales (
TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type]): Rank-2TileTensorof B scale factors. - epilogue_fn (
EpilogueFnType): Stores each output value at its global(m, n). - ctx (
DeviceContext): Device context used to enqueue the kernel launch.
def matmul_sm100_blockwise_scaled_fp8[ComputeFnType: ElementwiseComputeFn, //, *, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B](c: TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], compute_fn: ComputeFnType, ctx: DeviceContext)
Enqueues a blockwise scaled FP8 GEMM on SM100 (see the other overload), storing compute_fn(idx, value) into c.
Parameters:
- ComputeFnType (
ElementwiseComputeFn): Type ofcompute_fn. - transpose_b (
Bool): Whether B is loaded transposed; must beTrue. - umma_shape (
Tuple[Int, Int, Int]): 3D MMA instruction shape(MMA_M, MMA_N, MMA_K). - block_tile_shape (
Tuple[Int, Int, Int]): 3D block tile shape(BM, BN, BK). - a_swizzle (
TensorMapSwizzle): TMA swizzle mode applied to A shared memory loads. - b_swizzle (
TensorMapSwizzle): TMA swizzle mode applied to B shared memory loads.
Args:
- c (
TileTensor[Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Rank-2 outputTileTensoraccumulating the scaled GEMM result. - a (
TileTensor[Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Rank-2TileTensorof A operands withfloat8_e4m3fnelements. - b (
TileTensor[Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): Rank-2TileTensorof B operands withfloat8_e4m3fnelements. - a_scales (
TileTensor[Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type]): Rank-2TileTensorof A scale factors. - b_scales (
TileTensor[Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type]): Rank-2TileTensorof B scale factors. - compute_fn (
ComputeFnType): Transforms each output value before it is stored. - ctx (
DeviceContext): Device context used to enqueue the kernel launch.