IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_grouped_quantize_dynamic_block_scaled_with_row_indices

struct Struct_grouped_quantize_dynamic_block_scaled_with_row_indices

Registers the quantization.grouped.quantize.dynamic.block.scaled.with_row_indices graph op with the graph compiler.

Row-gather variant of quantization.grouped.quantize.dynamic.block.scaled: output row i quantizes input row row_indices[i] // row_index_divisor, which fuses the MoE expert permutation gather into the quantize loads. MoE passes the routing slot order straight from moe.create.indices with the top-k as the divisor, so no kernel runs between the two to map slots to tokens. A separate op carries the extra operand because graph ops have no optional inputs, and the fusion lives in the kernel because prologue fusion absorbs only elementwise and view producers, not gathers. Fold it back into the base op once the graph compiler fuses gathers generically.

Implemented traitsโ€‹

AnyType, Deinitable, Movable

Methodsโ€‹

executeโ€‹

static def execute[row_index_divisor: Int](output: T, scales: T, input: T, row_offsets: T, scales_offsets: T, expert_ids: T, sf_tensor: T, row_indices: T, context: DeviceContext)

Was this page helpful?