For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_grouped_quantize_dynamic_block_scaled_with_row_indices
struct Struct_grouped_quantize_dynamic_block_scaled_with_row_indices
Registers the quantization.grouped.quantize.dynamic.block.scaled.with_row_indices graph op with the graph compiler.
Row-gather variant of quantization.grouped.quantize.dynamic.block.scaled: output
row i quantizes input row row_indices[i] // row_index_divisor, which
fuses the MoE expert permutation gather into the quantize loads. MoE
passes the routing slot order straight from moe.create.indices with
the top-k as the divisor, so no kernel runs between the two to map slots
to tokens. A separate op carries the extra operand because graph ops have
no optional inputs, and the fusion lives in the kernel because prologue
fusion absorbs only elementwise and view producers, not gathers. Fold it
back into the base op once the graph compiler fuses gathers generically.
Implemented traitsโ
Methodsโ
executeโ
static def execute[row_index_divisor: Int](output: T, scales: T, input: T, row_offsets: T, scales_offsets: T, expert_ids: T, sf_tensor: T, row_indices: T, context: DeviceContext)