Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does stemming harm precision in text classification?

I have read stemming harms precision but improves recall in text classification. How does that happen? When you stem you increase the number of matches between the query and the sample documents right?

like image 807
samsamara Avatar asked Mar 02 '26 01:03

samsamara


1 Answers

It's always the same, if you raise recall, your doing a generalisation. Because of that, you're losing precision. Stemming merge words together.

On the one hand, words which ought to be merged together (such as "adhere" and "adhesion") may remain distinct after stemming; on the other, words which are really distinct may be wrongly conflated (e.g., "experiment" and "experience"). These are known as understemming errors and overstemming errors respectively.

Overstemming lowers precision and understemming lowers recall. So, since no stemming at all means no over- but max understemming errors, you have a low recall there and a high precision.

Btw, precision means how many of your found 'documents' are those you were looking for. Recall means how many of all 'documents', which were correct, you received.

like image 143
Nicolas Avatar answered Mar 04 '26 09:03

Nicolas