IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

RMSNormQuantizeDynamicScaledFloat8

struct RMSNormQuantizeDynamicScaledFloat8

Registers the quantization.rms_norm_quantize_dynamic_scaled_float8 graph op: mo.composite.reduce.rms_norm fused with a group-scaled (blockwise-FP8) mo.composite.quantize_dynamic_scaled_fp8. Same outputs and scale layout as the unfused pair; group_size_or_per_token must be a proper divisor of the row (per-token scales stay on the rms_norm + quantize composite).

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[input_type: DType, //, group_size_or_per_token: Int, multiply_before_cast: Bool](output: T, scales: T, input: T, gamma: T, epsilon: Float32, weight_offset: Scalar[input_type], scale_ub: Float32, ctx: DeviceContext)

Was this page helpful?