Cannot import name 'lemmatize' from 'gensim.utils' although I have installed Pattern

Question

I try to use lemmatize() function from gensim. They said I have to install pattern also to use this function. I have already install both gensim and pattern but every time I try to import lemmatize from gensim, it keeps showing this error

I use pip install gensim and pip install pattern to install the libraries. My gensim version is 4.0.1 and pattern is 3.6

Traceback (most recent call last):
  File "c:\Users\huynh\Desktop\machine_learning\test.py", line 1, in <module>
    from gensim.utils import lemmatize
ImportError: cannot import name 'lemmatize' from 'gensim.utils' (C:\Users\huynh\AppData\Local\Programs\Python\Python39\lib\site-packages\gensim\utils.py)

I tried to look up documentation about this but all I could find is that I have to install pattern to be able to use it. Does anyone have an idea why I still don't have lemmatize()? Thank you

Answer 1

Gensim only ever previously wrapped the lemmatization routines of another library (Pattern) – which was not a particularly modern or well-maintained option, so it was removed from Gensim-4.0.

Users can choose & apply their own lemmatization operations, if they think that's necessary, as a preprocessing step before applying Gensim's algorithms.

Some Python libraries offering lemmatization include:

• Pattern (Gensim's previously-included option, which can be used directly rather than through Gensim's old support): https://github.com/clips/pattern
• NLTK: https://www.nltk.org/api/nltk.stem.html#nltk.stem.wordnet.WordNetLemmatizer
• UDPipe: https://ufal.mff.cuni.cz/udpipe
• Spacy: https://spacy.io/api/lemmatizer
• Stanza: https://stanfordnlp.github.io/stanza/