IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

Struct_rope_quantize_dynamic_block_scaled

struct Struct_rope_quantize_dynamic_block_scaled

Registers the quantization.rope.quantize.dynamic.block.scaled graph op with the graph compiler.

Implemented traits​

AnyType, Deinitable, Movable

Methods​

execute​

static def execute[SF_VECTOR_SIZE: Int, head_dim: Int, inverse: Bool](output: T, scales: T, input: T, freqs: T, tensor_sf: Float32, context: DeviceContext)

Rotates each head's trailing columns by RoPE, then quantizes to a block-scaled format.

Parameters:

  • ​SF_VECTOR_SIZE (Int): Elements per block scale.
  • ​head_dim (Int): Columns per head.
  • ​inverse (Bool): Whether to rotate by the conjugate of each frequency.

Args:

  • ​output (T): The quantized input of shape [M, N] (or N // 2 packed FP4).
  • ​scales (T): The block scales in the 5D TCGEN interleaved layout.
  • ​input (T): The input of shape [M, N], a whole number of heads a row.
  • ​freqs (T): [M, rope_dim] interleaved (cos, sin) pairs.
  • ​tensor_sf (Float32): The tensor-wise scale factor.
  • ​context (DeviceContext): The device context.

Was this page helpful?