Plotting and Saving of all values on each iteration

Please note optimizers are not the simple creatures discussed in text book but they do a lot presolving, scaling and reformulation before solving. So it is not feasible convert algorithmic iterates to something meaningful for the user in general.

Note postsolve may only work properly on (near) optimal solutions and therefore what Mark suggests may not produce something useful. So be careful.

The optimizer has to be designed for get intermediate iterate out of the algorithm.