Hi @mcg ,
Finally, I wrote something and I d’ like to know your opinion. If there is something wrong, please let me know.
Mathematical formulation:
Let Y=[n \times 1] and X=[n \times p] be the response and independent variables, respectively, where n is the number of samples and p is the number of variables. In addition, the set of variables have been broken down into m subgroups, X^{(1)}, X^{(2)}, \ldots, X^{(m)} where each X^{(l)} is an n \times p_l matrices where p_l is the number of covariates in group l.
We aim to find \beta values and Y^{(0)} which minimize the below equation:
\begin{eqnarray}
\min_{\beta,Y^{0}} \bigg{ \big\lVert Y-\sum_{l=1}^{m} X^{(l)} \beta^{(l)} - Y^{0}\big\rVert_{2}^{2} + \lambda_1 \sum_{l=1}^{m} \sqrt{p_l} \big\lVert \beta^{l} \big\rVert_2 + \lambda_2 \big\lVert \beta^{(l)} \big\rVert_1 \bigg}
\end{eqnarray}
subject to \beta^{(l)} \leq 0.
Implementation in MATLAB using CVX:
m = size(unique(groups),1); % Number of groups
p = size(X,2); % Number of variables
n = size(X,1); % Number of samples
group_idx % is a logical matrix [m,p]; Each row is a group, columns are variables
not_group_idx = ~group_idx' % I use it in CVX statements
sqr_group_sizes % is a vector [m,1] square root of group sizes
cvx_begin
variables beta(p,m) Y0(1)
minimize( square_pos (norm (Y - sum (X * beta, 2 ) - Y0 *ones (n, 1) ) ) + l1 * norms( beta, 2, 1) * sqr_group_sizes + l2 * sum( abs( beta(:) ) ) )
subject to
beta <= 0
% Zero-Constraint for grouping
beta(not_group_idx) == 0
cvx_end
I am looking forward to hearing from you.
Thanks.