Hello, I want to express the rank of the matrix in the constraints, how to express this rank()
There are only very limited types of rank constraints which are convex and compatible with CVX. You can read about them in https://convexoptimization.com/TOOLS/0976401304.pdf . Warning, that book is not easy reading, and I can’t vouch that everything in it is correct.
A typical constraint such as rank(X) == 1 is non-convex, and sometimes is relaxed, i.e., ignored, in order to make a problem convex and formulatable in CVX. If you really need rank(X) == 1, that is non-convex, and you can try using YALMIP.