How to choose proper method for scipy.optimize.minimize?

Question

I was wondering how I can choose the best minimization method for scipy.optimize.minimize and how different the results may be?

I am trying to minimize the following expression (solve for g):

|a1.g.x + a2.g.x^3 - K|

Answer 1

Scipy has a lecture on Mathematical Optimization, where they have a section on choosing a minimization method. Snippet taken from that section:

Without knowledge of the gradient:

• In general, prefer BFGS or L-BFGS, even if you have to approximate numerically gradients. These are also the default if you omit the parameter method - depending if the problem has constraints or bounds
• On well-conditioned problems, Powell and Nelder-Mead, both gradient-free methods, work well in high dimension, but they collapse for ill-conditioned problems.

With knowledge of the gradient:

• BFGS or L-BFGS.
• Computational overhead of BFGS is larger than that L-BFGS, itself larger than that of conjugate gradient. On the other side, BFGS usually needs less function evaluations than CG. Thus conjugate gradient method is better than BFGS at optimizing computationally cheap functions.

With the Hessian:

• If you can compute the Hessian, prefer the Newton method (Newton-CG or TCG).

If you have noisy measurements:

• Use Nelder-Mead or Powell.

If I have interpreted your equation correctly, I think that either BFGS or L-BFGS might work for you.