Markov chain as a linear constraint


(OS) #1

Hi all,
I’m curious if there is a trick to make this problem convex.
I have a convex objective when the domain is a probability distribution on random variables X,Y,Z (all alphabets are finite).
I would like to represent a Markov relation, say X-Y-Z, between the random variables as a linear constraint so it fits the convex optimization model.
In case you are not familiar with Markov chains, the Markov X-Y-Z implies that p(x|y,z) = p(x|z) for all x,y,z.

Thanks in advance,
Oron


(Mark L. Stone) #2

You need to spell out more clearly what your problem is. I have no idea what your non-convex problem formulation is, and so I am not in a position to offer advice as to making it into a convex formulation which CVX can accept.

Do you have a discrete Markov chain with finite number (n) of states and an n by n one-step transition matrix P.? And if so, what constraint(s) are you trying to impose? What are your optimization variables?


(Michael C. Grant) #3

(Michael C. Grant) #4

I’m going to close this, because it really has nothing to do with CVX specifically, but more generally about convex optimization or applications thereof. You should be asking this question on more general purpose mathematics forums like Math StackExchange or OR Exchange.