IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

bmm_sm100_blockwise_scaled_fp8

def bmm_sm100_blockwise_scaled_fp8[c_type: DType, a_type: DType, b_type: DType, a_scales_type: DType, b_scales_type: DType, *, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_scaling_block_n: Int = Int(128)](c_: TileTensor[c_type, Engine=c_.Engine, address_space=c_.address_space, linear_idx_type=c_.linear_idx_type], a_: TileTensor[a_type, Engine=a_.Engine, address_space=a_.address_space, linear_idx_type=a_.linear_idx_type], b_: TileTensor[b_type, Engine=b_.Engine, address_space=b_.address_space, linear_idx_type=b_.linear_idx_type], a_scales_: TileTensor[a_scales_type, Engine=a_scales_.Engine, address_space=a_scales_.address_space, linear_idx_type=a_scales_.linear_idx_type], b_scales_: TileTensor[b_scales_type, Engine=b_scales_.Engine, address_space=b_scales_.address_space, linear_idx_type=b_scales_.linear_idx_type], ctx: DeviceContext)

Computes a batched blockwise scaled FP8 matrix multiplication on SM100 (Blackwell) hardware by constructing TMA descriptors for the inputs and scales and enqueuing the blockwise FP8 kernel per batch slice.

Parameters:

  • ​c_type (DType): Output tensor element dtype.
  • ​a_type (DType): LHS input tensor element dtype.
  • ​b_type (DType): RHS input tensor element dtype.
  • ​a_scales_type (DType): LHS scales tensor element dtype.
  • ​b_scales_type (DType): RHS scales tensor element dtype.
  • ​transpose_b (Bool): Whether the RHS input is transposed (must be True).
  • ​umma_shape (Tuple[Int, Int, Int]): UMMA instruction shape (m, n, k) used by the kernel.
  • ​block_tile_shape (Tuple[Int, Int, Int]): CTA tile shape (BM, BN, BK).
  • ​a_swizzle (TensorMapSwizzle): TMA swizzle mode for the LHS input tensor.
  • ​b_swizzle (TensorMapSwizzle): TMA swizzle mode for the RHS input tensor.
  • ​b_scaling_block_n (Int): N-direction scale block size for the RHS scales.

Args:

def bmm_sm100_blockwise_scaled_fp8[c_type: DType, a_type: DType, b_type: DType, a_scales_type: DType, b_scales_type: DType, EpilogueFnType: ElementwiseEpilogueFn, //, *, transpose_b: Bool, umma_shape: Tuple[Int, Int, Int], block_tile_shape: Tuple[Int, Int, Int], a_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_swizzle: TensorMapSwizzle = TensorMapSwizzle.SWIZZLE_128B, b_scaling_block_n: Int = Int(128)](c_: TileTensor[c_type, Engine=c_.Engine, address_space=c_.address_space, linear_idx_type=c_.linear_idx_type], a_: TileTensor[a_type, Engine=a_.Engine, address_space=a_.address_space, linear_idx_type=a_.linear_idx_type], b_: TileTensor[b_type, Engine=b_.Engine, address_space=b_.address_space, linear_idx_type=b_.linear_idx_type], a_scales_: TileTensor[a_scales_type, Engine=a_scales_.Engine, address_space=a_scales_.address_space, linear_idx_type=a_scales_.linear_idx_type], b_scales_: TileTensor[b_scales_type, Engine=b_scales_.Engine, address_space=b_scales_.address_space, linear_idx_type=b_scales_.linear_idx_type], epilogue_fn: EpilogueFnType, ctx: DeviceContext)

Same as the plain overload, storing the output through epilogue_fn.

The kernel never writes c_; only its shape is read.

Parameters:

  • ​c_type (DType): Output tensor element dtype.
  • ​a_type (DType): LHS input tensor element dtype.
  • ​b_type (DType): RHS input tensor element dtype.
  • ​a_scales_type (DType): LHS scales tensor element dtype.
  • ​b_scales_type (DType): RHS scales tensor element dtype.
  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn (inferred).
  • ​transpose_b (Bool): Whether the RHS input is transposed (must be True).
  • ​umma_shape (Tuple[Int, Int, Int]): UMMA instruction shape (m, n, k) used by the kernel.
  • ​block_tile_shape (Tuple[Int, Int, Int]): CTA tile shape (BM, BN, BK).
  • ​a_swizzle (TensorMapSwizzle): TMA swizzle mode for the LHS input tensor.
  • ​b_swizzle (TensorMapSwizzle): TMA swizzle mode for the RHS input tensor.
  • ​b_scaling_block_n (Int): N-direction scale block size for the RHS scales.

Args:

Was this page helpful?