IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

block_scaled_matmul_block_diagonal

def block_scaled_matmul_block_diagonal[c_type: DType, a_type: DType, b_type: DType, scales_dtype: DType, //, *, groups: Int, SF_VECTOR_SIZE: Int, pdl_level: PDLLevel = PDLLevel.ON](c: TileTensor[c_type, Engine=c.Engine, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[scales_dtype, Engine=a_scales.Engine, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[scales_dtype, Engine=b_scales.Engine, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], tensor_sf: Float32, ctx: DeviceContext)

Computes a block-diagonal block-scaled matmul on SM100.

Block g of groups multiplies A's columns [g * K, (g + 1) * K) by B's rows [g * N, (g + 1) * N) and writes C's columns [g * N, (g + 1) * N), where K = b.dim[1] and N = c.dim[1] / groups. The blocks run as one batched matmul over strided views, so A and C stay token-major and the scales stay in the layouts the dense quantize and the weights already use.

Parameters:

  • ​c_type (DType): Element type of c (inferred).
  • ​a_type (DType): Element type of a (inferred).
  • ​b_type (DType): Element type of b (inferred).
  • ​scales_dtype (DType): Element type of both scale tensors (inferred).
  • ​groups (Int): The number of diagonal blocks.
  • ​SF_VECTOR_SIZE (Int): Elements per scale factor.
  • ​pdl_level (PDLLevel): Programmatic dependent launch level.

Args:

Raises:

If no SM100 kernel config covers the shape.

Was this page helpful?