IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

QuantizeDynamicScaledFloat8Pow2Scales

struct QuantizeDynamicScaledFloat8Pow2Scales

Registers the quantization.quantize_dynamic_scaled_float8.pow2_scales graph op.

Same numerics as mo.composite.quantize_dynamic_scaled_fp8 with float8_e8m0fnu scales, but the power-of-two scales are written widened to float32, the layout a blockwise fp8 matmul reads. This saves the separate cast kernel after each quantize.

A custom op rather than an attribute on the composite, as with row_bounded.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[group_size_or_per_token: Int, amax_floor: Float32 = Float32("0")](output: T, scales: T, input: T, scale_ub: Float32, ctx: DeviceContext)

Was this page helpful?