How to solve the error Only scalar quadratic forms can be specified in CVX

You can try something similar to @stephen_boyd’s answer in How to handle nonlinear equality constraints?

But there is no guarantee it will converge to anything, let alone a local optimum, let alone a global optimum. Maybe you should just use a non-convex nonlinear optimizer.