How to use log_sum_exp

(Basem El Halawany) #1

I need to write the following expression in a constraint using log_sum_exp function:

ln [ const1+Sum(const2*exp(x_i) ]

Also, how to write : Sum(exp(x_i))
I tried norm(exp(x),1) but does not work


(Mark L. Stone) #2

ln [ const1+Sum(const2*exp(x_i) ] is not the log of sum of exponentials, and can not be expressed using log_sum_exp. In fact, it’s not even necessarily convex.

As for Sum(exp(x_i)), you can write it as sum(exp(x)) if x is a vector variable.

How to handle log[exp(exp(x))-1]
(Basem El Halawany) #3

I have tried some mathematical manipulation for it:
const1+Sum(const2*exp(x_i) = exp(ln(const1)) + sum(exp(ln(const2) +x_i))

let z = [ln(const1) , ln(const2) +x_1 , …,ln(const2) +x_N)]

log_sum_exp(z) is accepted by cvx

(Mark L. Stone) #4

How is log_sum_exp(z) equal to your original expression? As I wrote before, ln [ const1+Sum(const2*exp(x_i) ] is not necessarily even convex. You have produced an expression which CVX will accept, but it is not equal to your original expression.

If you think I’m wrong, assign (non-trivial) numerical values to all your variables, including constants, as MATLAB (not CVX) variables, and compute your original expression and your version with log_sum_exp. Show us a reproducible example with complete MATLAB code which does this. Note I am not saying there are no input values in which your expressions come out with the same numerical value.

(Basem El Halawany) #5

Kindly check the following code:
clc;clear all;
N = 1e3;
i_ctr =3;
%% c1,c2, and elements of x_i in my case is greater than zero
c1 = 0.01 + (10-0.01).*rand(1,N);
c2 = 0.01 + (10-0.01).*rand(1,N);
x_i= 0.01 + (10-0.01).*rand(i_ctr,N);

for ctr = 1:N
expression1(ctr) = log(c1(ctr)+sum(c2(ctr)*exp(x_i(:,(ctr)))));
%% Expression2
temp =[log(c1(ctr)) ];
for m=1:i_ctr
temp= [temp log(c2(ctr))+x_i(m,ctr)];

expression2(ctr)  = log_sum_exp(temp);

%% I noticed some rounding , which appears in somecases
error = sum((round(expression1)-round(expression2))>0)

I assumed i want to convert the constant values to equivalent exponential terms . I assumed:
c2 =exp(newterm2)
by taking ln of both sides, we get:
newterm2= ln(c2)
which I merged (multiplied) with individual elements in x_i
The same applied with c1

Let me know your feedback
Thanks for the discussions

(Mark L. Stone) #6

I am kind of confused by what you’re doing. Maybe we are solving different problems.

Consider the function
f1(x) = log(2-3*exp(x))
which is a one dimensional case of your original expression ln [ const1+Sum(const2*exp(x_i) ] .
The 2nd derivative of f1(x) is negative everywhere, so it is concave.

Now consider
f2(x) = log(2+3*exp(x))
Its 2nd derivative is positive everywhere, so it is convex.

So even in one dimension, your original expression is not necessarily convex, depending on the values of the constants. Am I misunderstanding what expression you want to use in CVX?

(Basem El Halawany) #7

yes you understand it right
I mentioned as a comment in my code that within my problem, C1,C2, and all elements of x is non-negative.
So this is similar to your second assumption which is true a convex.

however, if it is a general expression with the parameters can take any value, i agree with you that this will not work.
Sorry , I am just beginning to use cvx and I am really appreciate this forum and your discussions.
The DCP rule sets are still confusing me

(Mark L. Stone) #8

Because any log_sum_exp which CVX accepts is convex, then if your original expression can be converted into log_sum_exp which CVX accepts, it must necessarily be convex. Conversely, if your original expression is not convex, it can not be converted into a log_sum_exp which CVX accepts. So ln [ const1+Sum(const2*exp(x_i) ] can not be converted into a log_sum_exp which CVX will accept for a general const1 and const2. Is your transformation somehow only valid for const1 and const2 > 0 because you take log of const?

The bottom line is that if you have produced code which CVX accepts, then the question is whether that CVX code represents the problem you are trying to solve. You need to decide whether that CVX code (using log_sum_exp) represents the problem you want to solve.

I think I got confused (mistaken) on my post before my previous post by inadvertently not taking a log at the end.

(Michael C. Grant) #9

Yes, if C1 and C2 are nonnegative, then this is convex. It violates the DCP rules, however, because it’s relying on undocumented capabilities intended to help support the geometric programming capability of CVX.

The DCP rules really aren’t that difficult. They’re very simple, in fact. But they can be frustrating if you’re not using CVX properly. For instance, trying to get CVX to solve a non-convex problem, or even a convex problem that cannot be expressed using the DCP ruleset, can be frustrating. Even just trying to see “how far you can go” with CVX is challenging.

Some application areas don’t really lend themselves to effective use of CVX for one or more of these reasons.