I want to find a diagonal matrix P which should maximize the sum of the eigenvalues of the Hermitian matrix H*H’, obtained as shown below. The matrices G, H_ia, and H_ba are all complex and their dimensions are as indicated.

G = (n*m)
H_ia = (k*n)
H_ba = (k*m)
u = size(H_ia,2);
H = [];
cvx_begin %quiet
variable P(u,u) complex diagonal
for t = 1:1:size(H_ia,1)
H = [H; ((H_ia(t,:)*P*G) + H_ba(t,:))];
end
maximize(lambda_sum_largest(H*H', size(H_ba,1)));
subject to
abs(diag(P)) == 1;
cvx_end

Upon running, CVX fails at H*H’ and returns the error message below. Can anyone help with that?

Error using * (line 126)
Disciplined convex programming error:
Only scalar quadratic forms can be specified in CVX

Note: There has been a major edit of this answer, because originally when i read maximize, I was thinking minimize.

I’m not quite clear if you want the sum of all the eingenvalues of H*H', which is equal to trace(H*H'). if so

A little mathematical hocus pocus, and voila lambda_sum_largest(H*H',n)
where n is the dimension of H,
can be reformulated as square_pos(norm(H,'fro'))

But that is convex, and can’t be maximized in CVX.

abs(diag(P)) == 1; is a non-convex constraint, which I think you would have to handle by introducing binary variables.

Unless you meant minimize, not maximize, this problem is non-convex (due to the objective function), and can’t be handled by CVX.

Thank you @Mark_L_Stone for prompt assistance and suggestions. Yes, you got me right, I wanted the sum of all eigenvalues of H*H'.

Unfortunately, my model needs the maximization of these eigenvalues. Meanwhile, let me see if I can relax some aspects of the model so that it can be handled by CVX.

If you wanted to maximize the sum of all the eigenvalues of a hermitian semidefinite matrix J, you could do that in CVX as mazimize(trace(H)), because trace is affine (both convex and concave). Aa I showed, in CVX, you can minimize sum of thw eigenvalues of H*H' for any complex H, but you can’ not maximize it in CVX

If you really want to maximize that, try using YALMIP to invoke a local or global nonconvex solver. Minimize -trace(H*H'), that will avoid the considerable complication and performance degradation of dealing with eigenvalues.Together with the non-convex constraint, I believe this can be formulated as a non-convex MIQP.