Sharing objective and constraint calculation within Scipy Optimize Minimize() SLSQP

Question

I've made an app that performs multivariate constrained optimization using Scipy's minimize() function (using the SLSQP solver). It's working ok, so now I'm looking at improving the performance of it.

The process for calculating my objective value is quite heavy, and takes about a quarter of a second to compute. That doesn't sound like much, but when minimise() does its thing, it does this calculation about 600 times. Additionally, the process for running the constraint function takes about the same time, and is also computed about 600 times during optimisation.

The thing is, the code for calculating the objective value and the constraint value is nearly identical, just the very last bit at the end differs. (My code processes a set of variables, and produces a set of results. Some of those results are relevant for calculating the objective value, while others are relevant for calculating the constraint value).

Is there a way that I can share the bulk of the calculation between the objective value calculation and the constraint value calculation? If this is possible, then I could nearly halve my optimization time.

I've noticed that the constraints dictionary can have extra arguments 'args' passed to it. Perhaps there is a way of passing it the set of 'results' gained during the objective value calculation?

Thanks, Hugh.

Answer 1

First, what is the dimension of your problem ?

SLSQP uses very few function and constraints evaluations, so my guess is that most of the 600 evals you mention come from the gradient approximation code. So one way to make it faster (as @sascha already pointed out) is to pass the explicit gradient function to the solver, if (1) you can compute it and (2) this computation is faster than dimension times the time to compute the objective.

Now to answer the original question, I think the solution is caching. If your objective and constraint share most of the computation, then do it only once for both.

You can encapsulate this in a class, like this (Disclaimer: this is not running code):

class Model:

    def __init__(self):
        self.cache = {}

    def _in_cache(self, x):
        # search for x as key in self.cache
        return x in self.cache

    def _compute_solution(self, x):
        # do all your calculations only once here
        # in the end fill the cache
        self.cache[x] = {"objective": f, "constraint": g}  # f, g are scalars

    def objective(x):
        if not self.in_cache(x):
             self._compute_solution(x) # this fills the cache with objective and constraint values for x
        return self.cache[x]["objective"]

    def constraint(x):
        if not self.in_cache(x):
             self._compute_solution(x) # same idea
        return self.cache[x]["constraint"]

Of course the cache may grow to un-manageable size, so you may have to clean it sometimes, but 600 will not be a problem here. you may also assign a unique id to each x and use something like a look-up table, since I'm not sure if you can (and how efficient it is) to use arrays as dictionary keys.

This solution has the advantage that you can call any method first (the objective or the constraint) and also many times (until you clean the cache).