Generalized eigenvalue problem

Your problem is quasi-convex, so you can use the bisection method described in section 4.2.5 “Quasiconvex optimization” of Boyd and Vandenberghe “Convex Optimization” https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf . CVX can be used to solve the LMIs within the bisection algorithm.

Strict inequalities will be interpreted as non-strict inequalities, so because of the homogeneity, you’ll need to do something to prevent X = zero matrix from occurring as a solution (with any lamnda) So, as one way of doing so, force X to be positive definite, such as X - eye(size(x)) == semidefinite(size(X)). That won’t affect optimal lambda.