CVX on Octave: Discussion


(Michael C. Grant) #1

Now that Octave 4.0 has been released, the current version of CVX 3.0 beta has now achieved alpha-level stability on Octave. For more information, see this blog post on the CVX web site.

If you would like to discuss your experiences installing or using Octave, please do so on this post only. That way, everyone who is on the bleeding edge with you can keep in close contact. All other questions about CVX and Octave will be merged with this one.

Once CVX+Octave is more stable, we’ll open up the full forum.


#2

I want to test the current version of CVX 3.0 on Octave, but i fail to install CVX on my Windows 7 64-bit machine.
I have downloaded the file cvx-w64.zip and ran the following commands:

cd C:\octave\cvx
cvx_setup

It reports:

  ---------------------------------------------------------------------------
    CVX: Software for Disciplined Convex Programming       (c)2014 CVX Research
    Version 3.0beta, Build 1173 (c9b179b)               Sat Aug 1 15:20:04 2015
    ---------------------------------------------------------------------------
    Installation info:
        Path: C:\octave\cvx
        GNU Octave 4.0.0 on i686-w64-mingw32
    Verfying CVX directory contents:
        No missing files.
    ---------------------------------------------------------------------------
    ERROR: the following CVX MEX files are missing:
        C:\octave\cvx\lib\o_win\cvx_eliminate_mex.mex
        C:\octave\cvx\lib\o_win\cvx_classify_mex.mex
        C:\octave\cvx\lib\o_win\cvx_bcompress_mex.mex
    CVX will not operate without these files. Please visit
        http://cvxr.com/cvx/download
    and download a distribution built for this platform.
    ---------------------------------------------------------------------------
        ERROR: -------------------------------
        Error detected by cvx_version
            Line 22: C:\octave\cvx\cvx_setup.m
        --------------------------------------
    The installation of CVX was not completed. Please correct the error
    and re-run CVX_SETUP.
    ---------------------------------------------------------------------------

What am I doing wrong?


(Michael C. Grant) #3

You have to compile the MEX files yourself. If you’ve downloaded the most recent beta, I think we have a ‘cvx_compile’ function you can try.


#4

Thanks! That resolved the error, but now I am getting another unexpected error:

⁣---------------------------------------------------------------------------
CVX: Software for Disciplined Convex Programming       (c)2014 CVX Research
Version 3.0beta, Build 1173 (c9b179b)               Sat Aug 1 15:20:04 2015
---------------------------------------------------------------------------
Installation info:
    Path: C:\octave\cvx
    GNU Octave 4.0.0 on i686-w64-mingw32
Verfying CVX directory contents:
    WARNING: The following extra files/directories were found:
        C:\octave\cvx\lib\cvx_bcompress_mex.mex
        C:\octave\cvx\lib\cvx_bcompress_mex.o
        C:\octave\cvx\lib\cvx_classify_mex.mex
        C:\octave\cvx\lib\cvx_classify_mex.o
        C:\octave\cvx\lib\cvx_eliminate_mex.mex
        C:\octave\cvx\lib\cvx_eliminate_mex.o
    These files may alter the behavior of CVX in unsupported ways.
Loading preferences:
    Global: C:\octave\cvx\cvx_prefs.mat ... not found.
    Local: C:\Users\personal\.cvx_prefs.mat ... to be created.
---------------------------------------------------------------------------
Setting CVX paths...done.
Saving update path...done.
Searching for solvers...5 shims found.
3 solvers initialized (* = default):
    GLPK     unknown    C:\Octave\Octave-4.0.0\share\octave\4.0.0\m\optimization
 *  SDPT3    4.0        {cvx}\sdpt3
    SeDuMi   1.34       {cvx}\sedumi
2 solvers not found:
    ECOS     https://github.com/ifa-ethz/ecos
    SCS      https://github.com/cvxgrp/scs
Saving updated preferences...done.
Testing with a simple model...
    UNEXPECTED ERROR: --------------------------------------
    'mexnnz' undefined near line 97 column 13
        Line 97: C:\octave\cvx\sdpt3\Solver\checkdepconstr.m
    --------------------------------------------------------
Please report this error to support, and include entire output of
CVX_SETUP in your support request.
---------------------------------------------------------------------------

(Michael C. Grant) #5

Looks like SDPT3’s MEX files aren’t compiled.


(Alessandro Cosentino) #6

I cloned the latest master from Github, but I cannot find cvx_compile. Is it supposed to be still there?


(Michael C. Grant) #7

Ah, good point, you need to clone rework, which is the CVX 3.0beta branch. Sorry for the confusion.


(Alessandro Cosentino) #8

Thanks, I cloned the rework branch and I found the cvx_compile command. Everything went fine, except I manage to compile only the SDPT solver. The install_sedumi gives the following error:

Attempting to recompile the SeDuMi binaries:
Template: mex -O -DOCTAVE <sources> -lblas
   bwblkslv.mex: bwblkslv.c sdmauxFill.c sdmauxRdot.c
In file included from bwblkslv.c:45:0:
blksdp.h:41:21: fatal error: f77blas.h: No such file or directory
 #include "f77blas.h"

Anyway, that’s fine, I can use SDPT for my application. I tried to run the SDP I am solving and I get the following error:

error: concatenation operator not implemented for '<unknown type>' by 'sparse complex matrix' operations
error: called from
    minus_nc at line 10 column 3
    cvx_binary_op at line 107 column 11
    cvx_pushcnstr at line 116 column 13
    ge at line 22 column 3
    ydy at line 9 column 12

This is a MWE (very minimal, not much sense for that SDP):

cvx_begin sdp
    cvx_quiet(false);

    variable P(2,2) hermitian;
    
    maximize(trace(P));

    subject to
        P >= 0;
cvx_end

(Michael C. Grant) #9

I’m afraid that’s one of the exact bugs that prevents me from moving forward with a formally supported CVX release. From the blog post I linked to above: Complex variables will not work due to a known bug in Octave 4.0. I’ve reported this bug to the Octave folks and I’m sure it will get fixed in a subsequent release.


(Alessandro Cosentino) #10

I was wondering if there is a way to impose everything to be real. In theory, that shouldn’t change my SDP.


(Michael C. Grant) #11

Well yes, declare your SDP symmetric instead of Hermitian. Hermitian implies complex.


(Alessandro Cosentino) #12

I had tried symmetric, but that gives a strange value, which is not the same I get with MATLAB. I guess it’s not quite working yet even if I impose everything to be real. Anyway, it’s great to see progress on CVX on Octave, i am looking forward to porting everything to it once it’s ready :wink:


(Michael C. Grant) #13

If you can offer a simple example that fails, please go ahead and submit a bug report. I definitely need some test cases where Octave fails and Matlab does not.


(Alessandro Cosentino) #14

Do you want me to open an issue on Github? Or where should I report the bug?


(Michael C. Grant) #15

I’d say go to http://support.cvxr.com … Thanks!


(Alessandro Cosentino) #16

Hi Michael,

After stripping out all useless parts, the program became very simple and I suspect that either I am doing something wrong or it’s already fixed, so I will just report it here:

cvx_begin sdp
  variable P(2,2) symmetric    
  maximize trace(P)
  subject to
      P == eye(2);
cvx_end

The variable cvx_optval is 2 on MATLAB and 1 on Octave. Let me know if I should open a ticket for it.


(Michael C. Grant) #17

No, this is awesome right here. Thanks!


(jbbacque) #18

There is now an existence proof that CVX 3.0 can be coerced into at least setting up correctly in Octave 4.0.0 running in a Cygwin environment. TBD if it all works, but I thought I would share a couple of bumps and point to “fixes” as I finally achieved setup success:

cd /cygdrive/c/octave/cvx
cvx_setup


CVX: Software for Disciplined Convex Programming ©2014 CVX Research
Version 3.0beta, Build 1175 (1326ef2) Mon Nov 23 14:29:34 2015

Installation info:
Path: /cygdrive/c/octave/cvx
GNU Octave 4.0.0 on x86_64-unknown-cygwin
Verfying CVX directory contents:
WARNING: The following extra files/directories were found:
/cygdrive/c/octave/cvx/lib/cvx_bcompress_mex.mex
/cygdrive/c/octave/cvx/lib/cvx_bcompress_mex.o
/cygdrive/c/octave/cvx/lib/cvx_classify_mex.mex
/cygdrive/c/octave/cvx/lib/cvx_classify_mex.o
/cygdrive/c/octave/cvx/lib/cvx_eliminate_mex.mex
/cygdrive/c/octave/cvx/lib/cvx_eliminate_mex.o
/cygdrive/c/octave/cvx/lib/mex.h
/cygdrive/c/octave/cvx/lib/mexproto.h
/cygdrive/c/octave/cvx/lib/mxarray.h
/cygdrive/c/octave/cvx/lib/ov-mex-fcn.h
(and 118 more files)
These files may alter the behavior of CVX in unsupported ways.
Loading preferences:
Global: /cygdrive/c/octave/cvx/cvx_prefs.mat … not found.
Local: /home/jbbacque/.cvx_prefs.mat … to be created.

Setting CVX paths…done.
Saving update path…done.
Searching for solvers…5 shims found.
3 solvers initialized (* = default):
GLPK unknown /usr/share/octave/4.0.0/m/optimization


To change the default solver, type “cvx_solver <solver_name>”.
To save this change for future sessions, type “cvx_save_prefs”.
Please consult the users’ guide for more information.

=============================================================================

Bumps:

  • make sure you have gcc c++ compiler installed (duh, but it was missing in my cygwin at first…)

  • [having now noticed there is an edit function here, I’ll pre-apologize for verbosity below and will edit out newbie stuff later, but still capturing issues as they are found and resolved here for now…]


(jbbacque) #19

(… and don’t hit “tab” in this editor!.. oops… continuing…)

  • look above at the “extra files” that cvx_setup warns about. These missing .h include files either need to be in the ~cvx\lib directory, or somehow inform octave and gcc where they can be found through a library environment variable or command-line option. I took the easy cheat, and simply copied missing .h files from wherever they could be found (typically
    C:\cygwin64\usr\include\octave-4.0.0\octave or
    C:\cygwin64\usr\src\debug\octave-4.0.0-3\libinterp\corefcn) to the location of the file that included it, typically c:/octave/cvx/…

  • make sure you have openBLAS and libopenblas in your cygwin config!

  • you will still likely find you are missing “f77blas.h” and/or other math support. I resolved this by fetching libopenblas-dev_0.2.15-1_i386.deb from github, then peeled out f77blas.h with 7-zip into c:/octave/cvx … (and perhaps some other files… it grows murky…) It seems this file is NOT part of the normal openblas distribution…

  • but I still couldn’t get a clean SeDuMi build - “cannot find –lblas” from the linker.

  • Hoping to pick up the required linker dll support without too much more digging around, I fleshed out the cygwin installed math support to include:
    lapack, libamd1, libarpack0, libcamd1, libccolamd1, libcholmod1, libcolamd1, libcxsparse0, libfftw3_3, libglpk36, libgmp10, liblapack-devel, liblapack-doc, liblapack0, libmetis0, libmpc3, libmpfr4, libopenblas, libqhull_6, libqrupdate0, libsuitesparseconfig1, libumfpack1, octave, octave-devel, octave-doc, octave-general, octave-miscellaneous, octave-optim, octave-struct, and openblas are all installed. All are sufficient, it seems, all may not be necessary.

  • I also installed python, python 3k, and numpy (I already had pulled in optim for octave, and lapack is also) as this seemed to pull in the right packages for others in other environments trying to do other things, but seeing the same issues.

Somewhere along the line the linker stopped complaining about the missing blas support and I got a clean build of both SeDuMi and SDPT3 as you can see above.

Apologies for all the missing details in there, but that is roughly the process that I followed to get both of the solvers configured for cvx 3.0 on octave 4.0.0 on cygwin.


(jbbacque) #20

FYI the stripped-down real, symmetric problem above produces same optval=1 for my install described above, invariant of which solver is selected.

Update: SeDuMi solver produced correct output on snp500 time-series example, I start to believe…

Update: all examples in quickstart seemed to produce sane results.

Update Dec 24: “quiet” option seems to not yet be fully implemented in Octave, at least when running SeDuMi, as the “Constraint Accepted” messages continue to flow, and if the pager is enabled requires keyboard input on all my iterations. I’ll see what I can dig up, but not truly blocking, yet… update: found “disp.m” in ~cvx/lib/@cvxcnst/disp.m and put in a conditional on cvx_quiet. Nope, still paging though blanks… TODO: fix later - just turn off the pager in octave with octave>> more off.

When GLPK is selected on my problem error message:
"‘quiet’ undefined near line 95 column 4
error: called from ‘cvx_glpk>solve’ in file /cygdrive/c/octave/cvx/shims/cvx_glpk.m near line 95, column 1"
modifying cvx_glpk.m shim to use “cvx_quiet” instead of “quiet” fixes this one.

The command “cvx_pause.m” seems not to exist in the disttibution… getting it from https://code.google.com/p/matlab-code-by-wayne/source/browse/branches/cvx/commands/cvx_pause.m line 12: s = cvx___.pause; reveals that error: structure has no member ‘pause’. TODO deal later…

Overall, despite a few hiccups to get it up and running and to understand CVX’s disciplined-programming approach, CVX on Octave seems to be working for me on a model-predictive receding-horizon control problem.

Thanks all!
ben