maximum likelihood and support vector complexity

Question

Can anyone give some references showing how to determine the maximum likelihood and support vector machine classifiers' computation complexity? I have been searching the web but don't seem to find a good docs that details how to find the equations that model the computation complexity of those classifier algorithms. Thanks

Answer 1

Support vector machines, and a number of maximum likelihood fits are convex minimization problems. Therefore they could in theory be solved in polynomial time using http://en.wikipedia.org/wiki/Ellipsoid_method.

I suspect that you can get much better estimates if you consider methods. http://www.cse.ust.hk/~jamesk/papers/jmlr05.pdf says that standard SVM fitting on m instances costs O(m^3) time and O(m^2) space. http://research.microsoft.com/en-us/um/people/minka/papers/logreg/minka-logreg.pdf gives costs per iteration for logistic regression but does not give a theoretical basis for estimating the number of iterations. In practice I would hope that this goes to quadratic convergence most of the time and is not too bad.