Iterate Lambda to find MSE in cvx using LASSO

(Oskar) #1

Thread name might be a bit misleading so i will try to clarify.
I’m trying to minimize the weight vector of a optimization problem to later on use it to predict outcome of a stock market. For now the code that uses cvx looks like this:

lambda = 0.08;
cvx_begin quiet
cvx_precision low
variable w(n)
minimize(sum_square(Xw - Y) + lambdanorm(w,1))

As you can see I set the value of lambda but i would like cvx to iterate over different lambdas to find the best fit.
When that is done i would also like to set different lambdas inside the norm term so i can penalize specific weights instead of penalizing the whole weight vector, how do i go about doing that?

If anything is unclear, ask and i will try to clarify further!

Thanks in advance!

Best regards, Oskar

(Mark L. Stone) #2

For your first question, use a for or while loop around the CVX code. For instance, see

For the second question, you could use an n by 1 lambda vector. If you want to brute force the search, as with your first question, it will be quite computationally demanding (curse of dimensionaity) to loop over n dimensions. Therefore, a more sophisticated approach, perhaps as a non-convex nonlinear optimization problem, for which CVX can not be used, might be the best way to go. There is a lot of literature and software for multivariate hyperparameter optimization, which is what you are wanting to do.