I've recently noticed that a function where I iterate over a DataFrame rows using .iloc is very slow. I found out that there's a faster method called .iat, that's said to be equivalent to .iloc. I tried it and it cut the run time down by about 75%.
But I'm a little hesitant: why is there an "equivalent" method that's faster? There must be some difference between the inner workings of these two and a reason why they both exist and not just the faster one. I've tried looking everywhere but even the pandas documentation just states that
DataFrame.iat
Fast integer location scalar accessor.Similarly to iloc, iat provides integer based lookups. You can also set using these indexers.
And that doesn't help.
Are there limits to using .iat? Why is faster; is it sloppier? Or do I just switch to using .iat and happily forget .iloc ever existed?
at selects a single scalar value in the DataFrame by label only. . iat selects a single scalar value in the DataFrame by integer location only.
The iat method in pandas is used to get a single value from a dataframe based on the row and column index. The iat function needs two parameters: the row and column index. It then returns the value of the specified index.
loc can take multiple rows and columns as input arguments. . at can only take one row and one column as input arguments.
The main difference between pandas loc[] vs iloc[] is loc gets DataFrame rows & columns by labels/names and iloc[] gets by integer Index/position. For loc[], if the label is not present it gives a key error. For iloc[], if the position is not present it gives an index error.
iat and at gives only a single value output, while iloc and loc can give multiple row output.
Example: iloc[1:2,5:8] is valid but iat[1:2,5:8] will throw error
iat and at working with scalar only, so very fast. Slower, more general functions are iloc and loc.
You can check docs:
Since indexing with [] must handle a lot of cases (single-label access, slicing, boolean indexing, etc.), it has a bit of overhead in order to figure out what you’re asking for. If you only want to access a scalar value, the fastest way is to use the at and iat methods, which are implemented on all of the data structures.
Similarly to loc, at provides label based scalar lookups, while, iat provides integer based lookups analogously to iloc.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With