For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
blockwise_scaled_fp8_matmul_amd_batched
def blockwise_scaled_fp8_matmul_amd_batched[c_type: DType, a_type: DType, b_type: DType, a_scales_type: DType, b_scales_type: DType, //, *, scales_granularity_mnk: IndexList[Int(3)], transpose_b: Bool = False](c_: TileTensor[c_type, Engine=c_.Engine, address_space=c_.address_space, linear_idx_type=c_.linear_idx_type], a_: TileTensor[a_type, Engine=a_.Engine, address_space=a_.address_space, linear_idx_type=a_.linear_idx_type], b_: TileTensor[b_type, Engine=b_.Engine, address_space=b_.address_space, linear_idx_type=b_.linear_idx_type], a_scales_: TileTensor[a_scales_type, Engine=a_scales_.Engine, address_space=a_scales_.address_space, linear_idx_type=a_scales_.linear_idx_type], b_scales_: TileTensor[b_scales_type, Engine=b_scales_.Engine, address_space=b_scales_.address_space, linear_idx_type=b_scales_.linear_idx_type], ctx: DeviceContextBase) -> Int
Enqueues ONE batched blockwise-scaled FP8 dense matmul for gfx950.
This is the batched counterpart of blockwise_scaled_fp8_matmul_amd: the
same tile config and the same per-tile body, but grid.z = batch issues
the whole [B, M, N] batch as a single dispatch.
Reachability is gated by the caller (_batched_matmul_dynamic_scaled_fp8_ amd_tiled), which mirrors blockwise_scaled_fp8_with_epilogue's AMD gate
and falls back to the per-batch path for shapes this kernel cannot tile
(e.g. non-fp8 inputs or an unsupported output dtype).
Args:
- c_ (
TileTensor[c_type, Engine=c_.Engine, address_space=c_.address_space, linear_idx_type=c_.linear_idx_type]): Rank-3 output[B, M, N], innermost column stride 1 (asserted). The batch and row strides may be runtime -- the MLA absorb output is a strided per-head slab -- and are preserved per-batch. An e4m3 output is the unit-scale FP8 staging of the absorbed Q for an FP8 latent cache: the fp32 accumulator is cast on store, the same.cast[c_type]()the naive kernel does, so no epilogue. - a_ (
TileTensor[a_type, Engine=a_.Engine, address_space=a_.address_space, linear_idx_type=a_.linear_idx_type]): Rank-3 LHS[B, M, K], innermost stride 1, K the static trailing. - b_ (
TileTensor[b_type, Engine=b_.Engine, address_space=b_.address_space, linear_idx_type=b_.linear_idx_type]): Rank-3 RHS[B, N, K],transpose_b, innermost stride 1. - a_scales_ (
TileTensor[a_scales_type, Engine=a_scales_.Engine, address_space=a_scales_.address_space, linear_idx_type=a_scales_.linear_idx_type]): Rank-3[B, ceildiv(K,K_SCALE), M]. - b_scales_ (
TileTensor[b_scales_type, Engine=b_scales_.Engine, address_space=b_scales_.address_space, linear_idx_type=b_scales_.linear_idx_type]): Rank-3[B, ceildiv(N,N_SCALE), ceildiv(K,K_SCALE)]. - ctx (
DeviceContextBase): Device context.
Returns: