# How to write an expression involving entropy in CVX?

(asnfk) #1

Hi,

If I have n decision variables, I can easily write sum of entropies as -sum(entr(A)) where is A is the list of the variables. How should I write sum(entropy(A[i]/sum(A)))?

(Mark L. Stone) #2

That function is neither convex nor concave, and therefore will not be accepted by CVX.

Consider
f = (a*log(a)+b*log(b)) / (a+b)
The Hessian evaluated at a = b = 1 has eigenvalues -1/2 and 1/2, so is indefinite at that point.

Edit: I misread your parentheses. I’m guessing that the preceding part of my answer is what you really meant. Nevertheless, I will show the conclusion is the same with the parentheses literally as you wrote them.

Consider
f = a/(a+b)*log(a/(a+b)) + b/(a+b)*log(b/(a+b))

The Hessian evaluated at a = 1, b = 2 has eigenvalues -0.0171306101 and 0.3462526235, so is indefinite at that point.

(Pramesh Kumar) #4

Hi Mark! Can we write the following expression in cvx?
$$\sum_{i= 0}^{m} n_{i}*log(\frac{n_{i}}{\sum_{i}n_{i}})$$

This is a convex function.

(Mark L. Stone) #5

Consider
a*log(a/(a+b)) + b*log(b/(a+b))

Rewrite this as
-a*log((a+b)/a) - b*log((a+b)/b)
which equals
-a*log(1+b/a) - b*log(1+a/b)

help rel_entr
rel_entr(a,a+b) + rel_entr(b,a+b)