Tell scipy.optimize.minimize to fail

Question

I'm using scipy.optimize.minimize for unrestricted optimization of an objective function which receives a couple of parameters and runs a complex numerical simulation based on these parameters. This simulation does not always converge in which case I make the objective function return inf, in some cases, in others NaN.

I thought that this hack would prevent the minimization from converging anywhere near a set of parameters that makes the simulation diverge. Instead, I encountered a case where the simulation won't even converge for the starting set of parameters but instead of failing, the optimization terminates "successfully" with 0 iterations. It doesn't seem to care about the objective function returning inf.

Is there a way to tell scipy.optimize.minimize to fail, e.g. by raising some sort of exception. While in this case it's obvious that the optimization didn't terminate successfully - because of 0 iterations and the fact that I know the optimal result - at some point I want to run problems that I don't know the solution for and I need to rely on minimize to tell me if shit hit the fan. If returning lots of nans and infs doesn't "break" the algorithm I guess I'll have to do it by brute force.

---

Here is an example of what the almost-iteration looks like. The function - a function of two variables - is called 4 times over all:
1) at the starting point -> simulation diverges, f(x) = inf
2) at a point 1e-5 to the right (gradient approximation) -> simulation diverges, f(x) = inf
3) at a point 1e-5 higher (grad. appr.) -> simulation converges, f(x) = some finite value
4) once more at the starting point -> simulation diverges, f(x) = inf

Answer 1

You have 2 options I can think of:

• opt for constrained optimization
• modify your objective function to diverge whenever your numerical simulation does not converge. Basically this means returning a large value, large compared to a 'normal' value, which depends on your problem at hand. minimize will then try to optimize going in another direction

I am however a bit surprised that minimize does not understand inf as a large value, and does not try to look for a solution in another direction. Could it be that it returns with 0 iterations only when your objective function returns nan? You could try debugging the issue by printing the value just before the return statement in your objective function.