For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_rope_quantize_dynamic_block_scaled
struct Struct_rope_quantize_dynamic_block_scaled
Registers the quantization.rope.quantize.dynamic.block.scaled graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute[SF_VECTOR_SIZE: Int, head_dim: Int, inverse: Bool](output: T, scales: T, input: T, freqs: T, tensor_sf: Float32, context: DeviceContext)
Rotates each head's trailing columns by RoPE, then quantizes to a block-scaled format.
Parameters:
- SF_VECTOR_SIZE (
Int): Elements per block scale. - head_dim (
Int): Columns per head. - inverse (
Bool): Whether to rotate by the conjugate of each frequency.
Args:
- output (
T): The quantized input of shape[M, N](orN // 2packed FP4). - scales (
T): The block scales in the 5D TCGEN interleaved layout. - input (
T): The input of shape[M, N], a whole number of heads a row. - freqs (
T):[M, rope_dim]interleaved(cos, sin)pairs. - tensor_sf (
Float32): The tensor-wise scale factor. - context (
DeviceContext): The device context.