How can I solve this error in order to implement the CVX toolbox?about zGz type

Everything about your problem, both the objective and the constraint, are highly non-convex.

Expanding on @Michal_Adamaszek 's post:

You could consiider using YALMIP.

Maybe you can relax this formula using SDP and then using the Dinkelbach algorithm to solve it?