I am trying to solve the system of equations Ax=b, the matrix A is thin, rank(A) gives full column rank but the matrix A has very large condition number. using Pinv and mldivide gives me true estimates but when using CVX to solve for this LS problem I get something different. Why is this happening and how could it be fixed ?
probably CVX is treating it as an underdetermined sys of equations.
Perhaps you can show us the details, or at least the results and solver output? The readers of this forum have no idea how CVX is treating it because you haven’t told us what you did in CVX or shown us any code.
And what do you mean by “something different”? Is the optimal objective value (2-norm, or its square) significantly different, or just the argmin?
Ill-conditioned problems tend to have significant sensitivity and be numerically precarious. Is your CVX objective norm(A*x-b)? Or are you squaring it? Squaring ill-conditioned matrices is not good numerically.
Choice of solver might matter. Mosek or Gurobi might handle an ill-conditioned problem better than SeDuMi or SDPT3. And with Gurobi, you can crank up the NumericFocus parameter. You get some quadprecision action when NumericFocus = 3.
This is my program
cvx_begin
variable Theta_cvx(size(M_LS,2))
minimize norm(Y_all-M_LS*Theta_cvx)
cvx_end
both objective value and argmin are different. I tried all except Gurobi. Probably the quadprecision of Gurobi might help. How could I set the NumericFocus parameter as you stated.
“both objective value and argmin are different” is very vague. Different by how much? Please give lots of numerical details and solver log outputs.
I don’t think this is a cvx issue since this is such a simple and standard code. If you think some solvers have numerical problems then you can always contact that solver’s support (with a fully reproducible example with all input data etc.). First please study the solver’s log output for clues.