IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

block_scaled_matmul_with_epilogue

def block_scaled_matmul_with_epilogue[c_type: DType, a_type: DType, b_type: DType, scales_dtype: DType, EpilogueFnType: ElementwiseEpilogueFn = inflated$def[_no_epilogue_body[_, _, _, _, _]], ComputeFnType: ElementwiseComputeFn = inflated$def[identity_compute_fn[_, _, _, _, _]], //, *, SF_VECTOR_SIZE: Int, transpose_b: Bool = True, has_epilogue_fn: Bool = False, has_compute_fn: Bool = False](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[scales_dtype, Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[scales_dtype, Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], tensor_sf: Float32, ctx: DeviceContextBase, epilogue_fn: EpilogueFnType = no_epilogue_fn, compute_fn: ComputeFnType = no_compute_fn)

Our sm100 block scaled matmul kernel still does not support fusion of elementwise operations. This is a temporary implementation that uses our sm100 block scaled matmul kernel and dispatch a separate epilogue kernel to apply the elementwise operations. Callers must allocate c; with has_epilogue_fn or has_compute_fn the matmul result is written into c and then read back by the closure.

Parameters:

  • ​c_type (DType): Element type of the output matrix (inferred).
  • ​a_type (DType): Element type of the LHS input matrix (inferred).
  • ​b_type (DType): Element type of the RHS input matrix (inferred).
  • ​scales_dtype (DType): Element type of the block scale-factor tensors (inferred).
  • ​EpilogueFnType (ElementwiseEpilogueFn): Type of epilogue_fn (inferred).
  • ​ComputeFnType (ElementwiseComputeFn): Type of compute_fn (inferred).
  • ​SF_VECTOR_SIZE (Int): Number of elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8.
  • ​transpose_b (Bool): Whether b is stored transposed (defaults to True).
  • ​has_epilogue_fn (Bool): Whether epilogue_fn receives the matmul result after it is written to c.
  • ​has_compute_fn (Bool): Whether compute_fn transforms the matmul result in c.

Was this page helpful?