Issues using Gurobi and CVX: internal and external

I am trying to get Gurobi running with CVX. I have academic licenses for both of CVX and Gurobi, and am following the installation guide closely:

[EDIT by mcg: I’ve removed the setup scripts to save space; it’s clear the installation was performed properly, and Gurobi functions properly on the first attempt.]

So far so good! (The result itself is irrelevant.) But now I restart MATLAB and Gurobi is not working anymore:

% matlab
>> run /home/markus/work/matlab/cvx/cvx_startup.m
Setting CVX paths...done.
>> cd ~/temp/
>> hello
 
Calling Gurobi 5.60: 52 variables, 34 equality constraints
------------------------------------------------------------
------------------------------------------------------------
Status: Error
Optimal value (cvx_optval): NaN
 
Error using cvx_end (line 256)
UNEXPECTED ERROR: ---------------------------------------------
Invalid MEX-file
    '/home/markus/work/matlab/cvx/gurobi/a64/gurobi.mexa64':
    libgurobi56.so: cannot open shared object file: No such file
    or directory
Error in /home/markus/work/matlab/cvx/shims/cvx_gurobi.p>gurobi5
    (line 246)
Error in cvx_run_solver (line 46)
   [ varargout{1:nargout} ] = sfunc( inputs{:} );
Error in /home/markus/work/matlab/cvx/shims/cvx_gurobi.p>solve
    (line 442)
Error in cvxprob/solve (line 428)
            [ x, status, tprec, iters ] = shim.solve( At, b, c,
                cones, quiet, prec, solv.settings, eargs{:} );
Error in cvx_end (line 87)
        solve( prob );
Error in hello (line 10)
cvx_end
Please report this error to CVX Support by visiting
    http://support.cvxr.com/support/tickets/new
or by sending an email to cvx@cvxr.com. Please include the full
output of this function in your report. Thank you!
---------------------------------------------------------------


Error in hello (line 10)
cvx_end 
>>

Adding the path to Gurobi to LD_LIBRARY_PATH does not solve the problem:

% matlab
>> run /home/markus/work/matlab/cvx/cvx_startup.m
>> setenv('LD_LIBRARY_PATH',strcat(getenv('LD_LIBRARY_PATH'),':','/home/markus/work/matlab/cvx/gurobi/a64'))
>> cd ~/temp
>> hello
 
Calling Gurobi 5.60: 52 variables, 34 equality constraints
------------------------------------------------------------
------------------------------------------------------------
Status: Error
Optimal value (cvx_optval): NaN
 
Error using cvx_end (line 256)
UNEXPECTED ERROR: ---------------------------------------------
Invalid MEX-file
    '/home/markus/work/matlab/cvx/gurobi/a64/gurobi.mexa64':
    libgurobi56.so: cannot open shared object file: No such file
    or directory
[...]

Could you please tell me if I’m doing something wrong here.


EDIT: I can work around this by calling cvx_setup each time I start MATLAB, but this can’t be the solution, can it?


EDIT by mcg: I’m merging the two reports together. Here’s the second:

As I have some trouble getting the internal copy of Gurobi working, I am now trying to get an external Gurobi copy working with CVX. Gurobi itself works:

% matlab
>> cd ~/.gurobi/gurobi560/linux64/examples/matlab/
>> lp
Optimize a model with 2 rows, 3 columns and 4 nonzeros
Presolve removed 2 rows and 3 columns
Presolve time: 0.00s
Presolve: All rows and columns removed
Iteration    Objective       Primal Inf.    Dual Inf.      Time
       0    4.0000000e+00   0.000000e+00   0.000000e+00      0s

Solved in 0 iterations and 0.00 seconds
Optimal objective  4.000000000e+00

result = 

          status: 'OPTIMAL'
     versioninfo: [1x1 struct]
          objval: 4
         runtime: 0.0028
               x: [3x1 double]
           slack: [2x1 double]
              pi: [2x1 double]
              rc: [3x1 double]
          vbasis: [3x1 double]
          cbasis: [2x1 double]
       itercount: 0
    baritercount: 0
       nodecount: 0

     4

     1
     0
     1

Optimize a model with 2 rows, 3 columns and 4 nonzeros
Presolve removed 2 rows and 3 columns
Presolve time: 0.00s
Presolve: All rows and columns removed
Iteration    Objective       Primal Inf.    Dual Inf.      Time
       0    4.0000000e+00   0.000000e+00   0.000000e+00      0s

Solved in 0 iterations and 0.00 seconds
Optimal objective  4.000000000e+00
     4

     1
     0
     1

>>

Now I install and use CVX:

% pwd  
/home/markus/work/matlab
% tar xf ~/temp/cvx.tar.gz
% matlab
>> cd cvx
>> cvx_setup ~/temp/cvx_license.dat

---------------------------------------------------------------------------
CVX, version 2.0 (beta)                        (c) 2012, CVX Research, Inc.
Software for Disciplined Convex Programming
---------------------------------------------------------------------------
Version info:
    Code: build 1022, 2013-11-19 12:28:20
    Documentation: build 994, 2013-07-24 10:54:53
Installation info:
    Path: /home/markus/work/matlab/cvx
    MATLAB version: 8.1 (R2013a)
    OS: Linux amd64 version 3.12.2
    Java version: 1.6.0_17
Verfying CVX directory contents:
    No missing files.
License host:
    Username: markus
    Host ID: XXXXXXXXX
Installing license:
    File: ~/temp/cvx_license.dat
    Organization: XXX
    Contact: Markus XXX
    License type: academic
    Named user: markus
    Host ID: XXXXXXXXX
    Expiration: 2014-12-03 (365 days remaining)
    Status: verified
---------------------------------------------------------------------------
Setting CVX paths...done.
Saved preferences...not found; defaults created.
Searching for solvers...4 shims found.
3 solvers initialized (* = default):
    Gurobi   5.60       /home/markus/.gurobi/gurobi560
 *  SDPT3    4.0        {cvx}/sdpt3
    SeDuMi   1.34cvx    {cvx}/sedumi
1 solver skipped:
    Mosek    unknown    
        Could not find a MOSEK MEX file.
Saving updated preferences...done.
Saving updated path...failed. (see below)
Testing with a simple model...done!
---------------------------------------------------------------------------
To change the default solver, type "cvx_solver <solver_name>".
To save this change for future sessions, type "cvx_save_prefs".
Please consult the users' guide for more information.
---------------------------------------------------------------------------
NOTE: the MATLAB path has been changed to point to the CVX distribution. To
use CVX without having to re-run CVX_SETUP every time MATLAB starts, you
will need to save this path permanently. This script attempted to do this
for you, but failed---likely due to UNIX permissions restrictions.
To solve the problem, edit the file
    /home/markus/work/matlab/startup.m
and add the following line to the end of the file:
    run /home/markus/work/matlab/cvx/cvx_startup.m
Please consult the MATLAB documentation for more information about the
startup.m file and its proper placement and usage.
---------------------------------------------------------------------------
WARNING: CVX includes a function
    /home/markus/work/matlab/cvx/functions/square.m
that conflicts with a function of the same name found here:
    /opt/MATLAB/R2013a/toolbox/signal/signal/square.m
If you wish to use this second function, you will need to rename or delete
the CVX version, as it will likely produce different results. This will not
affect the use of SQUARE() within CVX models, because CVX relies on a
different, internal version of SQUARE() when constructing CVX models.
---------------------------------------------------------------------------

>> cvx_solver gurobi
>> cvx_solver

   Name     Status     Version   Location
------------------------------------------------------------------
   Gurobi   selected   5.60      /home/markus/.gurobi/gurobi560
   SDPT3    default    4.0       {cvx}/sdpt3
   SeDuMi              1.34cvx   {cvx}/sedumi

>> cvx_save_prefs
>> cd ~/temp/
>> hello
 
Calling Gurobi 5.60: 52 variables, 34 equality constraints
------------------------------------------------------------
Optimize a model with 34 rows, 52 columns and 290 nonzeros
Model has 11 quadratic constraints
Presolve removed 10 rows and 11 columns
Presolve time: 0.00s
Presolved: 34 rows, 41 columns, 270 nonzeros
Presolved model has 11 second-order cone constraints
Ordering time: 0.00s

Barrier statistics:
 AA' NZ     : 5.160e+02
 Factor NZ  : 5.950e+02
 Factor Ops : 1.368e+04 (less than 1 second per iteration)
 Threads    : 1

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0   9.02480610e+00  0.00000000e+00  7.03e+00 1.00e-01  4.92e-01     0s
   1   4.47848361e+00  3.15467905e+00  4.45e-01 1.00e-07  5.68e-02     0s
   2   4.47429871e+00  4.14219882e+00  5.33e-15 8.91e-11  6.51e-03     0s
   3   4.47400047e+00  4.47198700e+00  6.89e-13 9.47e-13  3.95e-05     0s
   4   4.47375137e+00  4.47374585e+00  4.32e-12 1.40e-11  1.08e-07     0s
   5   4.47375101e+00  4.47375098e+00  4.77e-10 7.85e-10  4.60e-10     0s

Barrier solved model in 5 iterations and 0.00 seconds
Optimal objective 4.47375101e+00

Warning: to get QCP duals, please set parameter QCPDual to 1



------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +4.47375
 
>>

So far so good (again…)! Now I quit and start MATLAB, and run hello again:

% matlab -nodesktop -nosplash
>> run /home/markus/work/matlab/cvx/cvx_startup.m
Setting CVX paths...done.
>> cd ~/temp
>> cvx_solver

   Name     Status             Version   Location
--------------------------------------------------------------------------
   Gurobi   selected,default   5.60      /home/markus/.gurobi/gurobi560
   SDPT3                       4.0       {cvx}/sdpt3
   SeDuMi                      1.34cvx   {cvx}/sedumi

>> hello
 
Calling Gurobi 5.60: 52 variables, 34 equality constraints
------------------------------------------------------------
Optimize a model with 34 rows, 52 columns and 290 nonzeros
Model has 11 quadratic constraints
Presolve removed 10 rows and 11 columns
Presolve time: 0.00s
Presolved: 34 rows, 41 columns, 270 nonzeros
Presolved model has 11 second-order cone constraints
Ordering time: 0.00s

Barrier statistics:
 AA' NZ     : 5.160e+02
 Factor NZ  : 5.950e+02
 Factor Ops : 1.368e+04 (less than 1 second per iteration)
 Threads    : 1

MKL ERROR: Parameter 13 was incorrect on entry to DGEMM .

At this point a pop-up opens:

MATLAB has encountered an internal problem and needs to close.

Details >> shows:

MATLAB crash file:/home/markus/matlab_crash_dump.22053-1:


------------------------------------------------------------------------
       Segmentation violation detected at Tue Dec  3 13:33:44 2013
------------------------------------------------------------------------

Configuration:
  Crash Decoding     : Disabled
  Current Visual     : 0x21 (class 4, depth 24)
  Default Encoding   : UTF-8
  GNU C Library      : 2.18 stable
  MATLAB Architecture: glnxa64
  MATLAB Root        : /opt/MATLAB/R2013a
  MATLAB Version     : 8.1.0.604 (R2013a)
  Operating System   : Linux 3.12.2 #9 SMP Mon Dec 2 08:27:46 CET 2013 x86_64
  Processor ID       : x86 Family 6 Model 42 Stepping 7, GenuineIntel
  Virtual Machine    : Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode
  Window System      : The X.Org Foundation (11404000), display :0.0

Fault Count: 1


Abnormal termination:
Segmentation violation

Register State (from fault):
  RAX = 0000000000000000  RBX = 00007f2b6e6b8280
  RCX = 00007f2b6e666fe0  RDX = 000c6b336e6670a0
  RSP = 00007f2be64b5cb0  RBP = 00007f2b6e6b8000
  RSI = 00007f2b6e6b7100  RDI = 0000000000000100

   R8 = 0000000000001380   R9 = 0000000000000200
  R10 = 0000000000000100  R11 = 0000000000000000
  R12 = 0003f95800000040  R13 = 0000000000000004
  R14 = 0018573b6e667160  R15 = 0000000000000100

  RIP = 00007f2b57937ac3  EFL = 0000000000010206

   CS = 0033   FS = 0000   GS = 0000

Stack Trace (from fault):
[  0] 0x00007f2b57937ac3              /opt/MATLAB/R2013a/bin/glnxa64/mkl.so+31701699 mkl_blas_avx_dgemm_kernel_0+00001827


This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.

If this problem is reproducible, please submit a Service Request via:
    http://www.mathworks.com/support/contact_us/

A technical support engineer might contact you with further information.

Thank you for your help.

Starting MATLAB, running cvx_setup (instead of run .../cvx_startup.m), then cvx_solver gurobi, cd ~/temp, hello works.

But starting MATLAB, running cvx_setup followed by cvx_solver gurobi, clear all, cd ~/temp/, hello crashes. I.e. the clear all makes it crash.

What is going on here?


~/temp/hello.m

m = 20; n = 10; p = 4;
A = randn(m,n); b = randn(m,1);
C = randn(p,n); d = randn(p,1); e = rand;
cvx_begin
    variable x(n)
    minimize( norm( A * x - b, 2 ) )
    subject to
        C * x == d
        norm( x, Inf ) <= e
cvx_end

Looks like a bug. I recommend submitting a report to http://support.cvxr.com. I’ll take a look at it independently of this, but submitting a report ensures that it’s on the schedule.

Thank you for both of the detailed reports! It looks like there are some dynamic library issues involving CVX, Gurobi, and MATLAB.

I would request that you submit a report to http://support.cvxr.com. I’ll take a look at it independently of this, but submitting a report ensures that it’s on the schedule. It also allows me to contact you directly, because I may need you to try some things on your local copy.

Presumably, this particular issue is limited to the Linux platform.

I have a suspicion, by the way, that your problems with external Gurobi are related to an issue I encounted with MOSEK on 64-bit Linux. The key is that the crash seems to be occurring in MATLAB’s MKL library.

MKL ERROR: Parameter 13 was incorrect on entry to DGEMM

means an incorrect MKL correctly. GuRoBi most likely have to fix something.

Indeed, I have already been in contact with Gurobi about this. They’re having the struggles you know about very well. :slight_smile: