Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

usecols to be case insensitive in pandas

Tags:

python

pandas

I am trying to cycle through large number of CSVs with same column names, however some of them have a different case.

I am using "usecols" in my code which throws an error when the case does not match mycols list.

df=pd.read_csv(fname,sep=",", encoding="ISO-8859-1",quotechar='"',error_bad_lines=False,dtype=object,usecols=mycols,index_cols=False)

How do I make sure that usecols ignores the case?

Thanks

like image 858
failsafe100 Avatar asked Oct 30 '25 06:10

failsafe100


1 Answers

From the pandas documentation (describing the callable version of usecols)

If callable, the callable function will be evaluated against the column names, returning names where the callable function evaluates to True. An example of a valid callable argument would be lambda x: x.upper() in ['AAA', 'BBB', 'DDD']. Using this parameter results in much faster parsing time and lower memory usage.

like image 85
Igor Rivin Avatar answered Nov 01 '25 20:11

Igor Rivin