# How to deal with "out of space" error when using Mosek to solve a conic optimization problem?

The objective matrix is a 490*490 large real PSD matrix. The detailed information of this problem is put up below. What reasons cause such an error? Are there some solutions to fix it?

cvx real affine expression (490x490 matrix, 120295 d.o.f.)

## Calling Mosek 9.1.9: 120814 variables, 78307 equality constraints

MOSEK Version 9.1.9 (Build date: 2019-11-21 11:34:40)
Platform: Windows/64-X86

Problem
Name :
Objective sense : min
Type : CONIC (conic optimization problem)
Constraints : 78307
Cones : 0
Scalar variables : 519
Matrix variables : 1
Integer variables : 0

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries : 1 time : 0.00
Lin. dep. - tries : 1 time : 0.02
Lin. dep. - number : 0
Presolve terminated. Time: 0.05
Problem
Name :
Objective sense : min
Type : CONIC (conic optimization problem)
Constraints : 78307
Cones : 0
Scalar variables : 519
Matrix variables : 1
Integer variables : 0

Optimizer - solved problem : the primal
Optimizer - Constraints : 78307
Optimizer - Cones : 0
Optimizer - Scalar variables : 519 conic : 0
Optimizer - Semi-definite variables: 1 scalarized : 120295
Factor - setup time : 1212.72 dense det. time : 0.00
Factor - ML order time : 798.44 GP order time : 0.00
Factor - nonzeros before factor : 3.07e+09 after factor : 3.07e+09
Factor - dense dim. : 0 flops : 1.60e+14
ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME
0 1.0e+00 1.8e+00 2.2e+00 0.00e+00 -3.200000000e+00 0.000000000e+00 1.0e+00 1213.17
Optimizer terminated. Time: 1234.91

Interior-point solution summary
Problem status : UNKNOWN
Solution status : UNKNOWN
Primal. obj: 0.0000000000e+00 nrm: 0e+00 Viol. con: 1e+00 var: 0e+00 barvar: 0e+00
Dual. obj: 0.0000000000e+00 nrm: 0e+00 Viol. con: 0e+00 var: 0e+00 barvar: 8e-01
Optimizer summary
Optimizer - time: 1234.91
Interior-point - iterations : 1 time: 1234.86
Basis identification - time: 0.00
Primal - iterations : 0 time: 0.00
Dual - iterations : 0 time: 0.00
Clean primal - iterations : 0 time: 0.00
Clean dual - iterations : 0 time: 0.00
Simplex - time: 0.00
Primal simplex - iterations : 0 time: 0.00
Dual simplex - iterations : 0 time: 0.00
Mixed integer - relaxations: 0 time: 0.00

## Mosek error: MSK_RES_ERR_SPACE (Out of space.)

Status: Error
Optimal value (cvx_optval): NaN

That means you ran out of memory. Possible solutions are run on a bigger machine or buy more RAM.

It will reduce memory consumption to some extent if you run on 1 thread (set MSK_IPAR_NUM_THREADS to 1 in cvx solver options or set MSK_IPAR_INTPNT_MULTI_THREAD to 0).

Thank you very much for your advise. I think I need to check exactly how much memory is being used when my progamme is running.

You have 1 matrix variable. Now if that appears in almost all constraints then Mosek needs to work with a dense matrix of size

78307 times 78307

when forming the Schur complement. That requires a large amount of storage.

To implement @Michal_Adamaszek 's advice, use `'cvx_solver_settings( '{name}', {value} ) ` as follows
`cvx_solver_settings('MSK_IPAR_NUM_THREADS',1)`
or
`cvx_solver_settings('MSK_IPAR_INTPNT_MULTI_THREAD'.0)`

help cvx_solver_settings

current solver being used by CVX. Before using this function, please

``````cvx_solver_settings( <name1>, <value1>, [ <name2>, <value2>, ... ] )
stores a custom setting called <name1> for the active solver, and gives
it a value of <value2>, and so forth for the other key-value pairs.
Each name must be a string containing a valid variable name; the values
may be of any type or size, including empty arrays or strings.

When CVX_END is reached, these key/value pairs will be delivered to the
underlying solver as parameters. This allows an expert user to modify
the behavior of the underlying solver in a manner that is otherwise
unavailable through more standard CVX commands.

cvx_solver_settings( 'name' ) or
cvx_solver_settings name
returns the current value of that setting for the current solver. If
there is no such setting by that name, an error results.

cvx_solver_settings( '-clear', <name1>, '-clear', <name2>, ... ) or
cvx_solver_settings -clear <name>
Removes the key/value pair matching <name> from the settings list for
the active solver.

cvx_solver_settings, with no arguments, displays the current list of
settings for the active solver.

cvx_solver_settings -all displays a list of the settings for all
all available solvers.

cvx_solver_settings -clear clears all settings for the active solver.

cvx_solver_settings -clearall clears all settings for all solvers.

cvx_solver_settings( S )
where S is a structure, replaces the *entire* list of solver settings
with the field-value pairs stored in the structure S.

S = cvx_solver_settings returns a structure containing all of the
settings for the current solver.

If you call cvx_solver_settings within a model --- that is, between
CVX_BEGIN and CVX_END --- then the changes are "local": that is, they
will apply ONLY to the current model.

On the other hand, if you call cvx_solver_settings outside of a model,
then the changes are "global": they will apply to all subsequent models
that employ that solver.

CVX_SAVE_PREFS will save any global settings you have provided, so they
will be restored the next time you start MATLAB.

cvx_solver_settings( 'dumpfile', <filename> ) is a setting supported by
all solvers. If set, it will save a .MAT file containing the exact
input arguments delivered to the solver. This file will be created
immediately before the solver is called, so you will be able to examine
their contents even if the solver fails with an error. This feature is
to be used primarily by solver developers.

**** IMPORTANT NOTE ****
Please use this feature with extreme caution, and at your own risk:
* CVX does not check the correctness of the settings you supply. If the
solver rejects the setting you supply, CVX will fail until you change
or remove that setting.
* Use of this feature can alter the quality of the solutions that the
solver produces: sometimes for the better, sometimes for the worse.
* The settings set here *override* any default values CVX has chosen
for each solver. In certain cases, this may actually confuse CVX and
and cause it to misinterpret the results. Fpr this reason, we cannot
support all possible combinations of custom settings.``````

Thanks for the detailed instruction, I will try later.

Hi, thank you very much for the answer. So I suppose that most of the memory is used to store such a huge Schur complement matrix, then I have some questions:

1. How many Schur complements are needed to solve such a problem? Only one or multiple?
2. How much memory is enough to deal with it according to your estimation?My computer has memory of 64GB, is that enough?
3. Is such a Schur complement neccassery to solve my problem? Are there some special settings in Mosek or some other possible solvers which does not require such a large storage or can the algorithm in Mosek be refined to tackle such a problem?
I really appereciate it if you can give me some help if it’s convenient for you.

I kept monitoring the use of my memory and I found that even at the peak of it there sre still 2.5 GBs remained, but the programme still failed and responded with the “out of space” error, so why mosek can’t solve it?

If you dump the problem to disk using the instructions at

https://docs.mosek.com/9.2/faq/faq.html