How do I generate observations between two dates using Pandas

Question

I have a dataset in the following format:

User ID   Start Date   End Date
1         '2000-01-01' '2000-03-01'
2         '2002-01-01' '2002-08-01'
...         ....         ....
10        '2003-03-01' '2004-01-01'

How do I generate a dataset with each date between the start date and the end date for each user as follows:

User ID   Start Date  End Date      Activity Date
1         '2000-01-01' '2000-03-01' '2000-01-01'
1         '2000-01-01' '2000-03-01' '2000-02-01' 
1         '2000-01-01' '2000-03-01' '2000-03-01' 
2         '2002-01-01' '2002-08-01'  '2002-01-01'
...         ....           ...           ...
2         '2002-01-01' '2002-08-01'  '2002-07-01'
2         '2002-01-01' '2002-08-01'  '2002-08-01'
...         ....           ...           ...
...         ....           ...           ...
10        '2003-03-01' '2004-01-01'  '2003-03-01'
10        '2003-03-01' '2004-01-01'  '2003-04-01'
...         ....           ...           ...
...         ....           ...           ...
10        '2003-03-01' '2004-01-01'  '2003-12-01'
10        '2003-03-01' '2004-01-01'  '2004-01-01'

Answer 1

Use pd.date_range to generate dates from your start date to your end date. I have set the frequency to 30 days by doing freq=30D - choose whatever convenient for you.

df['Activity Date'] = df.apply(lambda s: pd.date_range(s['Start Date'], s['End Date'], freq='30D').tolist(), 1)
df = df.explode('Activity Date')

---

   User ID Start Date   End Date Activity Date
0        1 2000-01-01 2000-03-01    2000-01-01
0        1 2000-01-01 2000-03-01    2000-01-31
0        1 2000-01-01 2000-03-01    2000-03-01
1        2 2002-01-01 2002-08-01    2002-01-01
1        2 2002-01-01 2002-08-01    2002-01-31
1        2 2002-01-01 2002-08-01    2002-03-02
1        2 2002-01-01 2002-08-01    2002-04-01
1        2 2002-01-01 2002-08-01    2002-05-01
1        2 2002-01-01 2002-08-01    2002-05-31
1        2 2002-01-01 2002-08-01    2002-06-30
1        2 2002-01-01 2002-08-01    2002-07-30
2       10 2003-03-01 2004-01-01    2003-03-01
2       10 2003-03-01 2004-01-01    2003-03-31
2       10 2003-03-01 2004-01-01    2003-04-30
2       10 2003-03-01 2004-01-01    2003-05-30
2       10 2003-03-01 2004-01-01    2003-06-29
2       10 2003-03-01 2004-01-01    2003-07-29
2       10 2003-03-01 2004-01-01    2003-08-28
2       10 2003-03-01 2004-01-01    2003-09-27
2       10 2003-03-01 2004-01-01    2003-10-27
2       10 2003-03-01 2004-01-01    2003-11-26
2       10 2003-03-01 2004-01-01    2003-12-26