How to save the set of dominated solutions while solving NSGA 2 in pymoo into a dataframe?

Question

I am trying to solve a multiobjective optimization problem with 3 objectives and 2 decision variables using NSGA 2. The pymoo code for NSGA2 algorithm and termination criteria is given below. My pop_size is 100 and n_offspring is 100. The algorithm is iterated over 100 generations. I want to store all 100 values of decision variables considered in each generation for all 100 generations in a dataframe.

NSGA2 implementation in pymoo code:

from pymoo.algorithms.nsga2 import NSGA2
from pymoo.factory import get_sampling, get_crossover, get_mutation

algorithm = NSGA2(
    pop_size=20,
    n_offsprings=10,
    sampling=get_sampling("real_random"),
    crossover=get_crossover("real_sbx", prob=0.9, eta=15),
    mutation=get_mutation("real_pm", prob=0.01,eta=20),
    eliminate_duplicates=True
)
from pymoo.factory import get_termination

termination = get_termination("n_gen", 100)

from pymoo.optimize import minimize

res = minimize(MyProblem(),
               algorithm,
               termination,
               seed=1,
               save_history=True,
               verbose=True)

What I have tried (My reference: stackoverflow question):

import pandas as pd
df2 = pd.DataFrame (algorithm.pop)
df2.head(10)

The result from above code is blank and on passing

print(df2)

I get

Empty DataFrame
Columns: []
Index: [] 

Answer 1

Glad you intend to use pymoo for your research. You have correctly enabled the save_history option, which means you can access the algorithm objects. To have all solutions from the run, you can combine the offsprings (algorithm.off) from each generation. Don't forget the Population objects contain Individual objectives. With the get method you can get the X and F or other values. See the code below.

import pandas as pd

from pymoo.algorithms.nsga2 import NSGA2 from pymoo.factory import get_sampling, get_crossover, get_mutation, ZDT1 from pymoo.factory import get_termination from pymoo.model.population import Population from pymoo.optimize import minimize

problem = ZDT1()

algorithm = NSGA2(
    pop_size=20,
    n_offsprings=10,
    sampling=get_sampling("real_random"),
    crossover=get_crossover("real_sbx", prob=0.9, eta=15),
    mutation=get_mutation("real_pm", prob=0.01,eta=20),
    eliminate_duplicates=True )

termination = get_termination("n_gen", 10)


res = minimize(problem,
               algorithm,
               termination,
               seed=1,
               save_history=True,
               verbose=True)

all_pop = Population()

for algorithm in res.history:
    all_pop = Population.merge(all_pop, algorithm.off)

df = pd.DataFrame(all_pop.get("X"), columns=[f"X{i+1}" for i in range(problem.n_var)])

print(df)

Another way would be to use a callback and fill the data frame each generation. Similar as shown here: https://pymoo.org/interface/callback.html