for example x’Ax / x’Bx A and B are Positive semidefinite matrix。
How to determine whether a function is convex
Study Convex Optimization – Boyd and Vandenberghe
Per Why isn't CVX accepting my model? READ THIS FIRST! you should be telling us, not asking us, whether a function id convex.
(x'*A*x)/(x'*B*x) where A and B` are positive semidefinite, is neither convex nor concave.