How to write an expression involving entropy in CVX?


(asnfk) #1

Hi,

If I have n decision variables, I can easily write sum of entropies as -sum(entr(A)) where is A is the list of the variables. How should I write sum(entropy(A[i]/sum(A)))?


(Mark L. Stone) #2

That function is neither convex nor concave, and therefore will not be accepted by CVX.

Consider
f = (a*log(a)+b*log(b)) / (a+b)
The Hessian evaluated at a = b = 1 has eigenvalues -1/2 and 1/2, so is indefinite at that point.

Edit: I misread your parentheses. I’m guessing that the preceding part of my answer is what you really meant. Nevertheless, I will show the conclusion is the same with the parentheses literally as you wrote them.

Consider
f = a/(a+b)*log(a/(a+b)) + b/(a+b)*log(b/(a+b))

The Hessian evaluated at a = 1, b = 2 has eigenvalues -0.0171306101 and 0.3462526235, so is indefinite at that point.


(Pramesh Kumar) #4

Hi Mark! Can we write the following expression in cvx?
$$\sum_{i= 0}^{m} n_{i}*log(\frac{n_{i}}{\sum_{i}n_{i}})$$

This is a convex function.


(Mark L. Stone) #5

Consider
a*log(a/(a+b)) + b*log(b/(a+b))

Rewrite this as
-a*log((a+b)/a) - b*log((a+b)/b)
which equals
-a*log(1+b/a) - b*log(1+a/b)

Please look at
help rel_entr

Now use rel_entr as described in Writing a constraint in DCP complient format
That results in
rel_entr(a,a+b) + rel_entr(b,a+b)

So I believe your expression can be created as

variable n(m+1)
yourexpression = sum(rel_entr(n,sum(n)-n))

Please check this to make sure I didn’t make any mistakes.