Hello all,
I am trying to solve a simple LP problem, but it has a lot of variables and nonzero values. Below is the output of my program. I was wondering if I can seek someone’s help to resolve this issue.
Calling Gurobi 7.52: 606489 variables, 403174 equality constraints
Gurobi optimizer, licensed to CVX for CVX
Academic license - for non-commercial use only
Optimize a model with 403174 rows, 606489 columns and 1602254662 nonzeros
Model has 46626 quadratic constraints
Coefficient statistics:
Matrix range [4e-07, 1e+00]
QMatrix range [1e+00, 1e+00]
Objective range [3e-05, 4e+02]
Bounds range [0e+00, 0e+00]
RHS range [1e+00, 1e+02]
Status: Error
Optimal value (cvx_optval): NaN
Error using cvx_end (line 267)
UNEXPECTED ERROR: ---------------------------------------------
Error using gurobi
Gurobi error 10001: Out of memory
Error in cvx_gurobi
Error in cvx_run_solver (line 50)
[ varargout{1:nargout} ] = sfunc( inputs{:} );
Error in cvx_gurobi
Error in cvxprob/solve (line 429)
[ x, status, tprec, iters ] = shim.solve( At, b, c,
cones, quiet, prec, solv.settings, eargs{:} );
Error in cvx_end (line 88)
solve( prob );
Error in BasicLP_P1 (line 163)
Error in matlab.internal.editor.evaluateCode
Please report this error to CVX Support by visiting
http://support.cvxr.com/support/tickets/new
or by sending an email to cvx@cvxr.com. Please include the full
output of this function in your report. Thank you!
Thank you for your help!
Are you using sparse matrix(ces) for the constraints? The 1602254662 non-zeros consumes 1602254662 * 8/1024^3 = 11.94 GB plus sparse storage overhead. I don’t know ho much more memory than that CVX/Gurobi need to solve the problem.
Can you scale down to a smaller size problem of this type and see how large a problem you can solve with CVX/Gurobi?
Perhaps there are more clever, memory-efficient formulations available for your problem, but you haven’t told us enough about your problem or formulation to get specific advice. It may also be that some approach other than CVX, such as directly calling the solver, will be able to handle this problem in a more economical manner memory-wise.
See if the following Gurobi faq answer helps. Gurobi parameters can be changed when calling from CVX by using cvx_solver_settings, as documented at http://cvxr.com/cvx/doc/solver.html#advanced-solver-settings
http://www.gurobi.com/support/faqs
|What can be done to avoid an out-of-memory condition?
- When possible, use the 64-bit version of Gurobi Optimizer on a 64-bit system, which can handle much more memory than a 32-bit system.
- For a continuous (LP, QP, SOCP) model, use the simplex algorithm (Method=1 or 0), which requires less memory than the barrier or concurrent algorithms.
- For a MIP that requires a large number of nodes, use node files, which are specially designed for optimization; they are more efficient than general-purpose virtual memory from the operating system. To use node files, set the NodefileStart parameter to a value like 0.5. Note that NodefileStart has no effect for continuous models or for MIP models that solve at or near the root node.
- Try decreasing the Threads parameter, since its default behavior uses all cores in the computer.
- Try Gurobi Compute Server or Gurobi Cloud to offload the computing to another machine that has more memory.
Ideally, the best solution is simply to have enough memory. Memory is relatively inexpensive, and we recommend having sufficient memory so that the model can be loaded and solved in physical memory (RAM).|
|
Hello Mark,
Thank you for your reply!
Are you using sparse matrix(ces) for the constraints? The 1602254662 non-zeros consumes 1602254662 * 8/1024^3 = 11.94 GB plus sparse storage overhead. I don’t know ho much more memory than that CVX/Gurobi need to solve the problem.
Yes, I am using sparse matrices for the constraints.
Can you scale down to a smaller size problem of this type and see how large a problem you can solve with CVX/Gurobi?
I should be able to scale it down. I just wanted to check its capacity to solve it without scaling it down.
Perhaps there are more clever, memory-efficient formulations available for your problem, but you haven’t told us enough about your problem or formulation to get specific advice.
My formulation is Ax<=b where matrix A is sparse. However, I am calling a new variable to calculate the value of Ax. For instance,
cvx_begin
variable y(size)
variable x(size)
minimize (sum(y))
s.t
y = Ax
y(1:index)<= b(1:index)
y(index+1:end) <= b(index+1:end)
cvx_end
It may also be that some approach other than CVX, such as directly calling the solver, will be able to handle this problem in a more economical manner memory-wise.
Do you mean using Gurobi directly without cvx? How can I formulate this problem with gurobi alone instead of cvx/gurobi? I think Gurobi alone has a different way of formulating the problem than cvx/gurobi, which I am not familiar with.
See if the following Gurobi faq answer helps. Gurobi parameters can be changed when calling from CVX by using cvx_solver_settings , as documented at http://cvxr.com/cvx/doc/solver.html#advanced-solver-settings
I will look into this.
Thank you so much for your help!
If you wish to call gurobi directly without going through CVX, read the gurobi documentation. I know that’s old-fashioned, but give it a try.