Hello. This forum has been very helpful, so thank you to all who contribute. I have a problem where I would like to minimize the differential entropy for a Gaussian distribution… which is proportional to H where
H = log( det( P ) )
and P is the covariance matrix of the Gaussian. Now, in terms of my optimization variable K (a matrix), P = K * A * K^T which is quadratic, in order to not mess with that, you can create a square-root factor of P such that P = S * S^T, or S = K * chol(A) assuming A is PSD (which it is). Therefore, now I can work with a linear form, instead of quadratic (which I know CVX likes); that is, letting B = chol(A),
H = log( det( K * B ) )
Now, I know that H is concave. Which means I can only maximize it with CVX. So, then I was considering using the inverse of the covariance matrix L = inv(P) which ends up making
H = log( det( inv(K * B) ) ) = log( 1/det(K * B) )
Now, I haven’t found anything in this forum, in Boyd’s Convex Optimization, or generally online about proving convexity/concavity of this constraint. I guess I would like some help from any convex optimization guru who has tried to examine relaxations for these kinds of things.
log( 1/det(K * B) ) = -log_det(K*B), which is convex and would be accepted by CVX if K*B is symmetric PSD. But K*B would not necessarily be symmetric PSD.
However, log(det(inv(K*A*K'))) = -2*log_det(K)-log(det(A)), which is convex and would be accepted by CVX. The constant term -log(det(A)) could be omitted from the objective
Perhaps I’m getting a little confused as to what your actual problem is. So please clarify if this does what you want.
Thanks Mark. I just realized I made a mistake (on my last line of math)… I am just plugging back in the original, concave function… in terms of L = inv(P) = inv(K * A * K')
H = log(det(P)) = log(det(K*A*K')) = - log(det(inv(P)) = -log(det(inv(K*A*K')))
The RHS of the last equality is concave still in K… which makes sense because I wasn’t fundamentally changing anything. I think if I want H to be convex in my optimization variable K, ill have to redefine K to be something that makes it convex.
If you’re actually interested, I can send you the model. I have successfully used this model in CVX for a while (with a different objective function) and am just now trying to use the differential entropy.
I thought you were changing to the inverse of the original variable. I.e., concave in original variable, convex in inverse of that variable. Of course, the rest of your model has to allow the inverse of the original variable to now be the variable.