IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_grouped_quantize_nvfp4_row_scaled

struct Struct_grouped_quantize_nvfp4_row_scaled

Registers the mo.grouped.quantize.nvfp4.row_scaled graph op with the graph compiler.

Quantizes grouped BF16 rows to NVFP4 with a global scale per row, taken from the row's amax inside the kernel, and returns the scales for the grouped matmul's a_row_scales. The result is bit-identical to the chain of a row amax, two BF16 roundings and mo.grouped.quantize.dynamic.block.scaled with a tensor-wise scale of 1. apply_relu2 quantizes relu(x) * relu(x) in place of x, with relu from nn.activations.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[out_dtype: DType, scales_type: DType, in_dtype: DType, //, scales_rank: Int, apply_relu2: Int, target: StringSpan[ImmStaticOrigin]](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], scales: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=scales.static_spec], row_scales: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=row_scales.static_spec], input: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=input.static_spec], row_offsets: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=row_offsets.static_spec], scales_offsets: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=scales_offsets.static_spec], context: DeviceContextBase)

Was this page helpful?