For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
blockwise_scaled_fp8_matmul_amd_batched_kernel
def blockwise_scaled_fp8_matmul_amd_batched_kernel[c_type: DType, a_type: DType, b_type: DType, a_scales_type: DType, b_scales_type: DType, accum_type: DType, c_layout: TensorLayout, a_layout: TensorLayout, b_layout: TensorLayout, a_scale_layout: TensorLayout, b_scale_layout: TensorLayout, *, BM: Int, BN: Int, BK: Int, WM: Int, WN: Int, MMA_M: Int, MMA_N: Int, MMA_K: Int, N_SCALE: Int, K_SCALE: Int, N: Int, K: Int](c: TileTensor[c_type, c_layout, MutAnyOrigin], a: TileTensor[a_type, a_layout, ImmutAnyOrigin], b: TileTensor[b_type, b_layout, ImmutAnyOrigin], a_scales: TileTensor[a_scales_type, a_scale_layout, ImmutAnyOrigin], b_scales: TileTensor[b_scales_type, b_scale_layout, ImmutAnyOrigin])
One launch for the whole batch of a blockwise-scaled FP8 dense matmul.
Mirrors the SM100 _bmm_sm100_blockwise_scaled_fp8_kernel structure on
AMD: block_idx.z selects the batch element, block_idx.y the M-tile,
block_idx.x the N-tile, so a batched [B, M, N] problem is one kernel
launch of grid (ceildiv(N,BN), ceildiv(M,BM), B) instead of B per-
batch launches. The per-tile MMA body is the existing dense body, reused by
calling _blockwise_scaled_fp8_matmul_amd_tile with per-batch 2D views.
c/a carry the compile-time trailing extent (N/K) into the tile
body as a static last dimension, so the tile body's comptime N = type_of(c).static_shape[1] and comptime K = type_of(a).static_shape[1]
still bind. Rows and row strides stay runtime: the MLA absorb output is
a strided per-head slab into the latent, so the rank-3 strides are
preserved per-batch via ptr_at_offset(Coord(batch, 0, 0)) (which respects
the rank-3 strides) and the 2D layout carries the row stride.