Support Vector Regression Implementation in CVX

1

Hello experts,

I am new to CVX. I might be overlooking the DCP set rules. But I do not manage to run the problem (related to support vector regression–please see the attachment) in CVX.

Updated (to make the software run):

clear variables
N   = 100; 
M  = 2;
w  = [1;-1];     % weights
x  = randn(2,N); % input data
e  = randn(1,N); % noise
y  =  (w.')*x + e;
y  = y.';

lambda = 0.9;
eps = 0.5;
C   = 1/lambda;

cvx_begin 
    variables a(N) a_hat(N)
    minimize( norm((sum(a-a_hat))*x, 2)  - ((a-a_hat)'*y) + eps * sum(a+a_hat) )
    subject to
    0 <= a  <= C;
    0 <= a_hat  <= C;
    sum(a) - sum(a_hat) == 0;
cvx_end

Your help will be highly appreciated.

svr_problem_formulation
svr_problem_formulation.png1438×400 36.2 KB

Additional questions: For some reasons, I can’t reply to Mark et al.
Instead of this cost " norm((sum(a-a_hat))*x, 2) ", I would like to implement with summation as given in the last part of eqn. (11.48) in the snapshot. Please suggest.

Thank you so much

2

I believe you want
((a-a_hat)'*y')
i.e., with transpose on y. That will at least be conformal, and accepted by CVX.

You also need to use == , not = for equality constraints, such as what you intended with sum(a) - sum(a_hat) = 0;

Those two changes will allow the program to run. I leave it to you to determine whether that solves the correct optimization problem. Perhaps you’re off by a factor of 1/2 on the norm term, as well as not having squared it, etc. You can determine whether these really matter to you.

3

Thank you for your help Mark!

4

Thank you again for your help, Mark. For some reasons, I could not reply. Now, it seems, I can do it.
Now, the program runs at least. I have one more question.

Instead of this cost " norm((sum(a-a_hat))*x, 2) ", I would like to implement with summation as given in the last part of eqn. (11.48) in the snapshot. Please suggest.

5

Your initial reply required moderator approval - perhaps an overactive spam filter or something.

You can apply square_pos to the norm, and then the multiplicative factor of 1/2.

6

I see. Thank you. I will try.

I don’t know where to ask, why my support-vector regression (SVR) implementation in CVX do not produce sensible results. I need to dig into it further. I need to compare it with the implementation in matlab toolboox.