How to reshape a python vector when some elements are empty

Question

I have a df with values:

   A   B   C   D
0  1   2   3   2        
1  2   3   3   9        
2  5   3   6   6        
3  3       6   7       
4  6           7      
5  2              

df.shape is 6x4, say

df.iloc[:,1] pulls out the B column, but len(df.iloc[:,1]) is also = 6

How do I "reshape" df.iloc[:,1]? Which function can I use so that the output is the length of the actual values in the column.

My expected output in this case is 3

Answer 1

You can use last_valid_index. Just note that since your series originally contained NaN values and these are considered float, even after filtering your series will be float. You may wish to convert to int as a separate step.

# first convert dataframe to numeric
df = df.apply(pd.to_numeric, errors='coerce')

# extract column
B = df.iloc[:, 1]

# filter to the last valid value
B_filtered = B[:B.last_valid_index()]

print(B_filtered)

0    2.0
1    3.0
2    3.0
3    6.0
Name: B, dtype: float64