Using an arbitrary smooth function as first argument in `tfocs_SCD`

According to the TFOCS documentation, the first argument of tfocs_SCD must be a prox-capable function. An answer of question 562) in this forum seems to suggest that we can have an arbitrary smooth_op function as first argument in tfocs_SCD. Is this true? What if the proximity operator of this function is not available / not easy to compute?

Good question. I asked Stephen to chime in since he is the author of that answer. I do know this: when TFOCS requires a prox-capable function, there really is no substitute. It must be supplied.

Very good question. Turns out I answered the other question (562) too hastily, and it’s not really correct. You are right, tfocs_SCD requires a prox-capable function as the first argument. The reason the response in 562 is partially correct is that smooth_quad can usually compute the prox (though if it has a non-diagonal quadratic term, it will usually be expensive). In particular, unlike tfocs.m, tfocs_SCD.m cannot benefit from any smoothness in your problem, since it solves the dual.

If you really cannot compute the proximity operator of your first term, then there’s not much you can do, but I urge you to post the question on the forum to see if someone else knows how to compute the prox. And as always, the first term can but included as one of the dualized variables (if you do this, then use the empty matrix [] in the first term). The advantage of this is that now you can separate out a linear operator, which is often what prevented you from taking the prox. The disadvantage is that the dual problem will have more variables and in general will converge a bit slower, so you shouldn’t dualize terms needlessly.

Thanks. I presume a weaker sufficient condition on the first argument of tfocs_SCD would be that its Fenchel conjugate is easily computable. As in eq. (2.5) of your paper, that would allow geting rid of expensive linear operators by first passing to the dual and then back to primal, right?

No, unfortunately it is not that weak. For our approach, we do need the prox (of either the function or it’s conjugate). in a more general approach, not following the tfocs_SCD, however you solve it, you would need to know something more: e.g. a prox, a gradient, or a sub-gradient