IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

QuantizeDynamicScaledFloat8RowBounded

struct QuantizeDynamicScaledFloat8RowBounded

Registers the quantization.quantize_dynamic_scaled_float8.row_bounded graph op.

Same numerics as mo.composite.quantize_dynamic_scaled_fp8, but the rows it touches stop at a count the GPU publishes rather than at the height of the input tensor. The EP MoE down projection needs this: its activation buffer is sized for the worst-case dispatch, the dispatch kernel writes the live row count into row_offsets, and the host never learns that count.

A custom op rather than an operand on the composite, whose RMS-norm and all-reduce fusion patterns read only its input and scale_ub.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[group_size_or_per_token: Int](output: T, scales: T, input: T, scale_ub: Float32, row_offsets: T, ctx: DeviceContext)

Was this page helpful?