Error in CVX mosek: Unrecognized field name 'sol'

1

Dear experts,
I encountered the following problem while using CVX mosek to solve convex optimization problems. How can I solve it？

K_i =

 1

------------------------------------------------------------------------------------------------------------------------用户K_i = 1
Starting parallel pool (parpool) using the ‘local’ profile …
Connected to the parallel pool (number of workers: 10).

ans =

 1

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 1
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

 2

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 2
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

 3

ans =

 4

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 4
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

 5

ans =

 6

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 6
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

 7

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 7
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

 8

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 8
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

 9

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 9
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

ans =

10

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 10
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 3
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

------------------------------------------------------------------------------------------------------------------------蒙特卡洛MC_i = 5
CVX Warning:
Models involving “log” or other functions in the log, exp, and entropy
family are solved using an experimental successive approximation method.
This method is slower and less reliable than the method CVX employs for
other models. Please see the section of the user’s guide entitled
The successive approximation method
for more details about the approach, and for instructions on how to
suppress this warning message in the future.

Calling Mosek 9.1.9: 99127 variables, 24600 equality constraints
For improved efficiency, Mosek is solving the dual problem.

NOTE: custom settings have been set for this solver.

MOSEK Version 9.1.9 (Build date: 2019-11-21 11:34:40)
Copyright (c) MOSEK ApS, Denmark. WWW: mosek.com
Platform: Windows/64-X86

Problem
Name :
Objective sense : min
Type : CONIC (conic optimization problem)
Constraints : 24600
Cones : 12
Scalar variables : 25399
Matrix variables : 18
Integer variables : 0

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries : 1 time : 0.00
Lin. dep. - tries : 1 time : 0.02
Lin. dep. - number : 0
Presolve terminated. Time: 0.11
Problem
Name :
Objective sense : min
Type : CONIC (conic optimization problem)
Constraints : 24600
Cones : 12
Scalar variables : 25399
Matrix variables : 18
Integer variables : 0

Optimizer - threads : 24
Optimizer - solved problem : the primal
Optimizer - Constraints : 24588
Optimizer - Cones : 12
Optimizer - Scalar variables : 25387 conic : 25368
Optimizer - Semi-definite variables: 18 scalarized : 148608
Factor - setup time : 150.39 dense det. time : 39.41
Factor - ML order time : 49.38 GP order time : 0.00
Factor - nonzeros before factor : 3.02e+08 after factor : 3.02e+08
Factor - dense dim. : 0 flops : 4.99e+12
ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME
0 3.2e+01 1.2e+00 1.4e+01 0.00e+00 5.535567339e+00 -6.969099991e+00 1.0e+00 150.81
Optimizer terminated. Time: 156.95

Interior-point solution summary
Problem status : UNKNOWN
Solution status : UNKNOWN
Primal. obj: 0.0000000000e+00 nrm: 0e+00 Viol. con: 7e+00 var: 0e+00 barvar: 0e+00 cones: 0e+00
Dual. obj: 0.0000000000e+00 nrm: 0e+00 Viol. con: 0e+00 var: 1e+00 barvar: 0e+00 cones: 0e+00
Optimizer summary
Optimizer - time: 156.95
Interior-point - iterations : 1 time: 156.95
Basis identification - time: 0.00
Primal - iterations : 0 time: 0.00
Dual - iterations : 0 time: 0.00
Clean primal - iterations : 0 time: 0.00
Clean dual - iterations : 0 time: 0.00
Simplex - time: 0.00
Primal simplex - iterations : 0 time: 0.00
Dual simplex - iterations : 0 time: 0.00
Mixed integer - relaxations: 0 time: 0.00

Mosek error: MSK_RES_ERR_SPACE (Out of space.)

Status: Error
Optimal value (cvx_optval): NaN

Calling Mosek 9.1.9: 99127 variables, 24600 equality constraints
For improved efficiency, Mosek is solving the dual problem.

NOTE: custom settings have been set for this solver.

MOSEK Version 9.1.9 (Build date: 2019-11-21 11:34:40)
Copyright (c) MOSEK ApS, Denmark. WWW: mosek.com
Platform: Windows/64-X86

Problem
Name :
Objective sense : min
Type : CONIC (conic optimization problem)
Constraints : 24600
Cones : 12
Scalar variables : 25399
Matrix variables : 18
Integer variables : 0

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries : 1 time : 0.00
Lin. dep. - tries : 1 time : 0.02
Lin. dep. - number : 0
Presolve terminated. Time: 0.08
Problem
Name :
Objective sense : min
Type : CONIC (conic optimization problem)
Constraints : 24600
Cones : 12
Scalar variables : 25399
Matrix variables : 18
Integer variables : 0

Optimizer - threads : 24
Optimizer - solved problem : the primal
Optimizer - Constraints : 24588
Optimizer - Cones : 12
Optimizer - Scalar variables : 25387 conic : 25368
Optimizer - Semi-definite variables: 18 scalarized : 148608
Factor - setup time : 150.64 dense det. time : 38.36
Factor - ML order time : 50.80 GP order time : 0.00
Factor - nonzeros before factor : 3.02e+08 after factor : 3.02e+08
Factor - dense dim. : 0 flops : 4.99e+12
ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME
0 3.2e+01 1.2e+00 1.4e+01 0.00e+00 5.535567339e+00 -6.969099991e+00 1.0e+00 151.14
Optimizer terminated. Time: 156.77

Interior-point solution summary
Problem status : UNKNOWN
Solution status : UNKNOWN
Primal. obj: 0.0000000000e+00 nrm: 0e+00 Viol. con: 8e+00 var: 0e+00 barvar: 0e+00 cones: 0e+00
Dual. obj: 0.0000000000e+00 nrm: 0e+00 Viol. con: 0e+00 var: 1e+00 barvar: 0e+00 cones: 0e+00
Optimizer summary
Optimizer - time: 156.77
Interior-point - iterations : 1 time: 156.77
Basis identification - time: 0.00
Primal - iterations : 0 time: 0.00
Dual - iterations : 0 time: 0.00
Clean primal - iterations : 0 time: 0.00
Clean dual - iterations : 0 time: 0.00
Simplex - time: 0.00
Primal simplex - iterations : 0 time: 0.00
Dual simplex - iterations : 0 time: 0.00
Mixed integer - relaxations: 0 time: 0.00

Mosek error: MSK_RES_ERR_SPACE (Out of space.)

{错误使用 cvx_mosek
无法识别的字段名称 “sol”。
Error cvx_mosek
Unrecognized field name ‘sol’

2

Regarding

Error cvx_mosek
Unrecognized field name ‘sol’

see Error in solving one problem and same mosek solver can be solved another problem, What is the reason? - #4 by Michal_Adamaszek

As you can see, this is due to

Mosek error: MSK_RES_ERR_SPACE (Out of space.)

As to why Mosek ran out of space, I’ll defer to the Mosek people on the problem size shown in the log.

How does the iteration which failed differ from the earlier iterations? Are there more variables and/or constraints?

I have no idea what printed this

Starting parallel pool (parpool) using the ‘local’ profile …
Connected to the parallel pool (number of workers: 10).

But if you have parallel threads, perhaos that is contributing to memory usage?

3

Hello!
The variables and constraints in the iteration which failed are more than that in the earlier iterations.
I use ‘parfor’ to parallelly compute multiple for-loops. Hence, the following content appears in the command line window that :
Starting parallel pool (parpool) using the ‘local’ profile …
Connected to the parallel pool (number of workers: 10).

I use the command ‘cvx_solver_settings(‘MSK_IPAR_NUM_THREADS’, 1)’ to limit the number of threads used by Mosek in each for-loop.
My computer has a memory capacity of 128GB, and when the code runs, it shows that it occupies about 80GB of memory, and then the error 'Unrecognized field name ‘sol’ ’ appears.

4

Well, there was not enough memory.

Perhaps if there were fewer parfor workers, you wouldn’t run out of memory.

https://www.mathworks.com/matlabcentral/answers/20596-out-of-memory-in-parfor-loop

5

The error message in Matlab code and your prompt are correct.
I separately tested the memory size required for the code to run once, and it is indeed quite large.
I would like to ask you how to reduce the memory size required for CVX Mosek code?

6

I will defer to the Mosek people to address that. You might have to wait a couple of days fro them to see this. But perhaps if you showed your code, including at least the input data values relevant to the problem dimensions, that would help them to make an informed assessment.

7

Running on 1 thread reduces memory consumption a bit. Otherwise there is not much you can do except running on a bigger machine, but note that the memory complexity grows very fast with the number of constraints and dimensional of an SDP. The problem looks very large in this respect.

First of all install Mosek 10 or 11 instead of the old 9.1 that comes bundled with cvx and try that.

If you want you can contact Mosek support with the saved task file of the problem and then we can say something more specific.

8

Hello!
My Mosek Version is as follows:
image

Is the installation package for Mosek 11 downloaded from the following link?
link: Mosek - Downloads

c0a5f8df1755c52fee85c1c2c7cced50
c0a5f8df1755c52fee85c1c2c7cced501540×749 20.3 KB

Can an academic license be applied to Mosek 11?

How can I remove the old version of Mosek and install the new version?

9

If you have Mosek 10.2.6 installed then simply use it. In the log output it clearly calls Mosek 9.1. You probably need to choose one of the many Mosek’s available with the correct cvx_solver. Mosek 11 will not make much difference here as the interior-point solver is pretty much the same.

As mentioned we will be happy if you save task file 1 Technical Issues — MOSEK FAQ 10.2.7 and share it with Mosek support. Then we can say something more and also how much memory we needed.

CVX is normally very good in deciding to go for dualization to send the most efficient version of the problem to Mosek, so I would think that if it runs out of memory then that’s just life due to the sizes of the problem. But we can always take a look.

10

Thank you! How can I call Mosek 10.2.6 in CVX to solve the problem?

The file path of ‘mosekopt.mexw6’ in CVX is:
E:\Matlab2022a\cvx\mosek\w64\mosekopt.mexw64

The file path of ‘mosekopt.mexw6’ in Mosek is:
E:\Matlab2022a\mosek\10.2\toolbox\r2017a\mosekopt.mexw64.

Should I replace ‘mosekopt.mexw64 in CVX’ with ‘mosekopt.mexw64 in Mosek’?

11

You should have Mosek 10 installed in Matlab with the path added to the Matlab path, but apparently you already do if mosekdiag showed it.

When you run cvx_setup it should detect it and show you a list of solvers with a few copies of Mosek, usually called Mosek, Mosek_2, etc.

Then you choose one with cvx_solver, for example cvx_solver Mosek_2 and so on.

You should not be discouraged by the fact that CVX will list all of them as Mosek9.1.9, this is just a bug in the cvx report. Just look at the paths printed in cvx_setup and then use the solver with the correct path to Mosek 10.

Then when you solve the problem you should see in the log something like

MOSEK Version 10.2.6 (Build date: 2024.......)
Copyright (c) MOSEK ApS, Denmark. WWW: mosek.com
Platform: Windows/64-X86
12

Thank you! I get it.