For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
LayerNormQuantizeDynamicScaledFloat8
struct LayerNormQuantizeDynamicScaledFloat8
Registers the mo.layer_norm_quantize_dynamic_scaled_float8 graph op: a biased layer_norm (float32 statistics, rounded to the input dtype) fused with a group-scaled (blockwise-FP8) mo.quantize_dynamic_scaled_float8. Same outputs and scale layout as the unfused pair; group_size_or_per_token must be a proper divisor of the row, and the row width must be static.
Implemented traitsโ
Methodsโ
executeโ
static def execute[group_size_or_per_token: Int](output: T, scales: T, input: T, gamma: T, beta: T, epsilon: Float32, scale_ub: Float32, ctx: DeviceContext)