Define parameter in pyomo from pandas DataFrame

Question

First time pyomo user here.

I have a function that defines a model

def define_problem(SET_gen, SET_time, SET_buses, demand):                       

    model = pyo.ConcreteModel()

    #Define sets
    model.SET_GEN   = pyo.Set(initialize = SET_gen) #Set of generators
    model.SET_TIME = pyo.Set(initialize = SET_time) #Set of hours
    model.SET_BUSES = pyo.Set(initialize = SET_buses)   #Set of buses

    #Define parameters
    model.DEMAND = pyo.Param(model.SET_BUSES, model.SET_TIME, initialize = demand_init)
...

The argument 'demand' in the function is a pandas DataFrame

The function demand_init is define as the following

def demand_init(model, bus, t, data = demand):
    if(bus in set(data.columns)):
        return data.loc[t,bus]
    return 0.0

It should define the parameter model.DEMAND for each hour and each bus as the corresponding 'cell' in the demand DataFrame, and 0 if the bus is not in the DataFrame. EDIT: Is defined outside the define_problem function.

But its not working. How can i define the parameters of my function from a pandas DataFrame?

EDIT: Thanks for the answers!

The demand data frame looks like this:

      Bus1  Bus10  Bus11  Bus12  ...     Bus6  Bus7  Bus8   Bus9
Hour                             ...                            
1      0.0   9.00   3.50   6.10  ...    11.20   0.0   0.0  29.50
2      0.0   7.34   2.85   4.97  ...     9.13   0.0   0.0  24.06
3      0.0   6.45   2.51   4.37  ...     8.03   0.0   0.0  21.14
4      0.0   5.78   2.25   3.92  ...     7.20   0.0   0.0  18.95
5      0.0   5.56   2.16   3.77  ...     6.92   0.0   0.0  18.22

[5 rows x 14 columns]

The 't' and the 'bus' that should get into the demand_init function are the numbers in the index and the names of the columns in the data frame. They are in the sets model.SET_HOURS and model.SET_BUSES respectively.

Answer 1

You seem to have this one covered, so I'm just providing a couple of suggestions:

It's going to make your life much easier to just call the columns 1,2 etc. and call the axis bus, instead of calling each columns "Bus1" etc.

from pyomo import environ as pye
import pandas as pd
import numpy as np
​
n_bus = 5
n_hours = 10
​
demand_df = pd.DataFrame(
    data = np.random.random(size=(n_hours, n_bus)),
    columns = np.arange(1, n_bus+1), 
    index = np.arange(1, n_hours+1))
​
demand_df = demand_df.rename_axis('hour', axis=0)
demand_df = demand_df.rename_axis('bus', axis=1)

Now the DataFrame looks like

>>> demand_df.head()
bus 1           2           3           4           5
hour                    
1   0.249303    0.244917    0.348141    0.559970    0.414997
2   0.803017    0.940600    0.474955    0.976134    0.185487
3   0.776821    0.940770    0.482725    0.510914    0.186607
4   0.705604    0.871578    0.154195    0.943887    0.913865
5   0.039853    0.978370    0.320563    0.923042    0.591475

An easy way to obtain a dictionary {(hour,bus):value} is by doing:

demand_d = demand_df.stack().to_dict()

Now, you seem to want to define 0 as a default value. There are three ways (from worst to best, imho):

• use a defaultdict:

from collections import defaultdict
demand_d =defaultdict(int, demand_df.stack().to_dict())

• make sure all the columns are filled with 0 (.fillna(0))
• define a default value for the parameter

model.DEMAND = pyo.Param(
    model.SET_BUSES, model.SET_HOURS, 
    initialize = demand_d,
    default = 0)

As a final note, AbstractModel might help greatly reducing the effort that goes into manual data ingestion.

Answer 2

I changed my aproach and solved it.

You can pass a dictionary to the Param function, so I changed the demand_init function to the following:

def demand_init(model, data):
    init = {}
    for t in model.SET_HOURS:
        for bus in model.SET_BUSES:
            if(bus in set(data.columns)):
                init[bus,t] = data.loc[t,bus]
            else:
                init[bus,t] = 0
    return init

And then, I defined the parameter like this:

INIT_demand  = demand_init(model, data = demand)
model.DEMAND = pyo.Param(model.SET_BUSES, model.SET_HOURS, initialize = INIT_demand)

Both the Hours set and the Buses set must be previously defined.

I hope this helps someone.