IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

block_scaled_matmul

def block_scaled_matmul[c_type: DType, a_type: DType, b_type: DType, scales_dtype: DType, EpilogueFnType: ElementwiseEpilogueFn = inflated$def[_no_epilogue_body[_, _, _, _, _]], ComputeFnType: ElementwiseComputeFn = inflated$def[identity_compute_fn[_, _, _, _, _]], //, *, SF_VECTOR_SIZE: Int, transpose_b: Bool = True, transpose_a: Bool = False, has_epilogue_fn: Bool = False, has_compute_fn: Bool = False, pdl_level: PDLLevel = PDLLevel.OFF, _trace_description: StringSpan[ImmStaticOrigin] = StringSpan(""), target: StringSpan[ImmStaticOrigin] = StringSpan("cpu")](c_device: TileTensor[c_type, Engine=c_device.Engine, linear_idx_type=c_device.linear_idx_type], a_device: TileTensor[a_type, Engine=a_device.Engine, linear_idx_type=a_device.linear_idx_type], b_device: TileTensor[b_type, Engine=b_device.Engine, linear_idx_type=b_device.linear_idx_type], a_scales_device: TileTensor[scales_dtype, Engine=a_scales_device.Engine, linear_idx_type=a_scales_device.linear_idx_type], b_scales_device: TileTensor[scales_dtype, Engine=b_scales_device.Engine, linear_idx_type=b_scales_device.linear_idx_type], tensor_sf: Float32, ctx: DeviceContextBase, epilogue_fn: EpilogueFnType = no_epilogue_fn, compute_fn: ComputeFnType = no_compute_fn)

Dispatches the SM100 block-scaled matmul for NVFP4 or MXFP8, selecting between the Mojo heuristic kernel and the vendor fallback.

For small M or MXFP8 shapes the Mojo heuristic_and_outliers_dispatch kernel is tried first; on a dispatch miss the vendor (cuBLASLt) block-scaled matmul is used as a fallback.

Parameters:

  • ​c_type (DType): Element type of the output matrix (inferred).
  • ​a_type (DType): Element type of the LHS input matrix (inferred).
  • ​b_type (DType): Element type of the RHS input matrix (inferred).
  • ​scales_dtype (DType): Element type of the block scale-factor tensors (inferred).
  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn (inferred).
  • ​ComputeFnType (ElementwiseComputeFn): Type of compute_fn (inferred).
  • ​SF_VECTOR_SIZE (Int): Number of elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8.
  • ​transpose_b (Bool): Whether b_device is stored transposed (defaults to True).
  • ​transpose_a (Bool): Whether a_device is stored transposed (defaults to False).
  • ​has_epilogue_fn (Bool): Whether epilogue_fn stores the output instead of the kernel writing c_device.
  • ​has_compute_fn (Bool): Whether compute_fn maps each output value before it is stored.
  • ​pdl_level (PDLLevel): Programmatic dependency launch level for the heuristic dispatch kernel (defaults to PDLLevel.OFF).
  • ​_trace_description (StringSpan[ImmStaticOrigin]): Extra label appended to the trace event for profiling (defaults to "").
  • ​target (StringSpan[ImmStaticOrigin]): Trace target string identifying the device in profiling output (defaults to "cpu").

Was this page helpful?