For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
RowMeanOfSquaresQK
struct RowMeanOfSquaresQK
Fused per-row mean of squares for two operands Q and K.
For q of shape [M, Nq] and k of shape [M, Nk] (sharing rows but with
possibly different column counts), computes out[m, 0] = mean_n(q[m,n]^2)
and out[m, 1] = mean_n(k[m,n]^2) into a [M, 2] output. The square and
accumulation always run in float32. This is a single-launch fusion of two
mo.reduce.row_mean_of_squares ops plus a concat, used for cross-head
QK-RMSNorm statistics under tensor parallelism.
Implemented traits
Methods
execute
static def execute[target: StringSlice[ImmStaticOrigin]](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], q: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=q.static_spec], k: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=k.static_spec], ctx: DeviceContext)
Executes the mo.reduce.row_mean_of_squares_qk graph op.
Parameters:
- target (
StringSlice[ImmStaticOrigin]): Compilation target string.
Args:
- output (
ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec]): Output tensor of shape[M, 2]receiving the per-row mean of squares forqandk. - q (
ManagedTensorSlice[IOSpec[_, _].Input, static_spec=q.static_spec]): Query tensor of shape[M, Nq]. - k (
ManagedTensorSlice[IOSpec[_, _].Input, static_spec=k.static_spec]): Key tensor of shape[M, Nk]. - ctx (
DeviceContext): Device context used to enqueue the kernel.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!