I am trying to solve the following optimization problem that trains a triple-class classifier. The training set has n = 120 examples, each with feat = 4 features. Xtrain_augm is a matrix of size n x (1+feat), where each line is 1 followed by the four features, and Ytrain_i is an indicator vector of size n x 1 that is 1 if and only if the training example is from class i (i=1,2).
cvx_begin
variables beta_01 beta_1(feat,1) beta_02 beta_2(feat,1)
maximize( Ytrain_1’* Xtrain_augm*[beta_01;beta_1]+Ytrain_2’* Xtrain_augm*[beta_02;beta_2] …
- sum(log_sum_exp([zeros(1,n); [beta_01;beta_1]’* Xtrain_augm’; [beta_02;beta_2]’* Xtrain_augm’])) );
cvx_end
The successive approximation method with SDPT3 gives the wrong solution. Telling CVX to use SeDuMi performs better, but still seems unreliable.
Is there native support for this kind of problem in the current version of CVX that does not require installing outside software e.g. CVXQUAD or mosek? If not, how would you recommend solving this problem?
Does CVX saying is has terminated with the optimum, but you say it is wrong? I don;t know whether that can happen, but perhaps it can. More frequently, its fails to converge and declares failure, even though CVXQUAD or native exponential cone solver might succeed.
The best way to handle this in CVX is CVX 2.2 + Mosek 9.x.(without CVXQUAD’s exponential.m replacement). This utilizes Mosek 9.x’s native exponential cone capability.
The second best way is CVXQUAD (very easy to install and free) . Your program only uses log_sum_exp, so doesn’t require reformulation for CVX. You just need to unzip the CVXQUAD zip file, add the requisite directory to your MATLAB path, and swap in CVXQUAD’s exponential.m file for CVX’s. See CVXQUAD: How to use CVXQUAD's Pade Approximant instead of CVX's unreliable Successive Approximation for GP mode, log, exp, entr, rel_entr, kl_div, log_det, det_rootn, exponential cone. CVXQUAD's Quantum (Matrix) Entropy & Matrix Log related functions
The next best way is to use CVX’s successive approximation method with the best solver you have. If you have MOSEK, don;t use CVXQUAD’s exponential.m replacement.
The wildcard option is to use CVX 3.0beta, without CVXQUAD, and with ECOS or SCS, which will use the solver’s native exponential cone capability. Unfortunately, CVX 3.0beta is riddled with bugs, so this is really a crap shoot. There have been many instances in which CVX 3.0beta delcares optimality, but the solution is wrong. SO caveta emptor.