Sparse PCA in TFOCS

Here’s one way to do it. I modified proj_psdUTrace.m to take in an extra trace term (since this is just a linear term). An extra linear term is quite common so we may take care of it nicely in a future release of TFOCS. For now, the modified proj_psdUTrace.m is in github, so head there to get the latest version.

First, setup the problem

rng(100);
d = 20;
X = randn(d);
R = X'*X; % Sigma
e = ones(d,1);
lambda = 0.5;

Now solve in CVX so we can verify we’re solving the right problem

cvx_begin
cvx_precision best
 variable X(d,d) semidefinite
 minimize( -R(:)'*X(:)+lambda*norm(X(:),1) )
 subject to
 trace(X)==1;
cvx_end
toc
X_CVX = X;

And the TFOCS solve:

mu = 1e-2;
largescale  = ( d > 200 );
is_real     = true;
obj= proj_psdUTrace( 1, largescale, is_real, [], -R );
linear = linop_vec( [d,d] );
affine = { linear };
dualProx    = proj_linf(lambda);
x0      = [];
z0      = [];
opts    = struct('printEvery',4,'tol',1e-10,'maxIts',500,'L0',1);
opts.errFcn     = @(f,d,p) norm(p-X_CVX,'fro')/norm(X_CVX,'fro');
opts.restart    = 50;
[x,out,optsOut] = tfocs_SCD( obj, affine, dualProx, mu, x0, z0, opts );

which gives the following output on my computer:

Auslender & Teboulle's single-projection method
Iter    Objective   |dx|/|x|    step      errors
----+----------------------------------+----------
4   | -6.82076e+01  2.53e-01  1.52e+00 | 1.06e-01
8   | -6.70352e+01  9.12e-02  2.32e+00 | 6.26e-02
12  | -6.67141e+01  4.09e-02  3.54e+00 | 2.97e-02
16  | -6.66041e+01  2.35e-02  5.40e+00 | 1.44e-02
20  | -6.65586e+01  1.25e-02  8.23e+00 | 7.41e-03
24  | -6.65371e+01  7.68e-03  1.25e+01 | 4.15e-03
28  | -6.65259e+01  5.42e-03  1.91e+01 | 2.42e-03
32  | -6.65197e+01  3.74e-03  2.91e+01 | 1.43e-03
...
100 | -6.65105e+01  2.10e-06  2.63e+01*| 2.39e-07
101 | -6.65105e+01  4.26e-05  2.93e+01 | 3.13e-08 | restarted
104 | -6.65105e+01  3.68e-08  9.84e+00*| 1.11e-08
108 | -6.65105e+01  6.13e-09  1.50e+01*| 1.97e-09
112 | -6.65105e+01  2.03e-09  1.04e+01*| 3.20e-10
116 | -6.65105e+01  5.92e-10  1.59e+01*| 2.86e-10
120 | -6.65105e+01  1.24e-10  2.42e+01*| 1.72e-10
123 | -6.65105e+01  9.59e-11  1.32e+01*| 1.71e-10
Finished: Step size tolerance reached