For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_rms_norm_quantize_mxfp8_q_kv
struct Struct_rms_norm_quantize_mxfp8_q_kv
Registers the quantization.rms_norm.quantize.mxfp8.q_kv graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute(output: T, scales: T, kv_output: T, input: T, q_gamma: T, kv_gamma: T, epsilon: Float32, context: DeviceContext)
RMS-normalizes the q and kv parts of each row, quantizing q to MXFP8.
Args:
- output (
T): The quantized q of shape[M, q_cols]. - scales (
T): The block scales of q in the 5D TCGEN interleaved layout. - kv_output (
T): The normalized kv of shape[M, kv_cols]. - input (
T): The input of shape[M, q_cols + kv_cols], q then kv. - q_gamma (
T): The q weight. - kv_gamma (
T): The kv weight. - epsilon (
Float32): The RMS norm epsilon. - context (
DeviceContext): The device context.