a*a' involves product terms which are disallowed by CVX, and might be non-convex.

Have you proved the problem is convex? Yf so, perhaps a reformulation for CVX is possible. If it is not convex, then it can’t be handled by CVX. You haven’t shown enough information on your problem, so i don’t whether the problem can be reformulated for CVX.

My a stands for the alpha in the picture and label stands for y. test is my function to return kernel function(In a form of matrix). I want to use the dot product of the matrix, and add all the entries together, then I can get the sum term which I want to minimize in the picture(minus to maximize).