Error on quadratic form

Perhaps you can help us out by telling us exactly which statement triggered the error message? If it involves trace, you might be able to use the reformulation
trace (A*A') = square_pos(norm(A,'fro')) , and use cyclic permutation invariance of trace to rearrange the argument of trace to get it in this form if needed.

If that does not handle your situation, your first task is to prove your optimization problem is convex (concave objective being maximized, and all constraints convex, all holding jointly with respect to the CVX variables (accounting for anything going on in the CVX expressions).