Here A is a pre-defined 57x57 matrix.
But MATLAB throws an error " Only scalar quadratic forms can be specified in CVX". Can cvx solve the optimization problem? Thank you!

This formulation violates CVX’s rules.norm, or the alternative,sun_square, both require their argument to be affine, which yours is not.

I don’t know whether there is a reformulation compliant with CVX’s rules, but I am not optimistic. Note that, in general, there is no matrix X such that A*X = P*A*P for a permutation matrix P, which appears to close off a possible angle of attack.

I offer a free virtual beer, to anyone of legal virtual drinking age, who comes up with a CVX solution. The problem can be accepted by various global MINLP solvers (not available in CVX), but whether they can produce a solution within memory and run-time constraints is another matter.