For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo function
grouped_matmul_dynamic_scaled_fp8
def grouped_matmul_dynamic_scaled_fp8[c_type: DType, a_type: DType, b_type: DType, a_scales_type: DType, b_scales_type: DType, a_offsets_type: DType, expert_ids_type: DType, //, input_scale_granularity: StaticString, weight_scale_granularity: StaticString, m_scale_granularity: Int, n_scale_granularity: Int, k_scale_granularity: Int, transpose_b: Bool = False, static_grid_z: Bool = False](c: TileTensor[c_type, Engine=c.Engine, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Engine=a.Engine, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Engine=b.Engine, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[a_scales_type, Engine=a_scales.Engine, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[b_scales_type, Engine=b_scales.Engine, linear_idx_type=b_scales.linear_idx_type], a_offsets: TileTensor[a_offsets_type, Engine=a_offsets.Engine, linear_idx_type=a_offsets.linear_idx_type], expert_ids: TileTensor[expert_ids_type, Engine=expert_ids.Engine, linear_idx_type=expert_ids.linear_idx_type], max_num_tokens_per_expert: Int, num_active_experts: Int, ctx: DeviceContext, decode_grid_m_cap: Int = Int(0), estimated_total_m: Int = Int(0))
TileTensor primary implementation of grouped_matmul_dynamic_scaled_fp8.
estimated_total_m is the caller's estimate of the routed rows in the
call (0 when unknown); on SM100 it picks the tile size.