Meaning of re.compile(r"[\w']+") in Python

Question

I am new to python and trying to work on big data code but not able to understand what the expression re.compile(r"[\w']+") means.Anyone has any idea regarding this?

This is the code that i m using.

from mrjob.job import MRJob
import re

WORD_REGEXP = re.compile(r"[\w']+")

class MRWordFrequencyCount(MRJob):

    def mapper(self, _, line):
        words = WORD_REGEXP.findall(line)
        for word in words:
            yield word.lower(), 1

    def reducer(self, key, values):
        yield key, sum(values)


if __name__ == '__main__':
    MRWordFrequencyCount.run()

Answer 1

This is a regular expression that has been compiled for faster reuse (explained in this question: Is it worth using re.compile). The command re.compile is explained in the Python docs.

Regarding the specific regex expression, this searches for groups that have alphanumerics (that's the \w part) or apostrophes (which is also in those square brackets) that are 1 or longer. Note that whitespace is not a match, so this, generally speaking, breaks a line into words.

See the query in a Python specific regex tester to try it out or on regex101 where they offer an explanation of any regex expression.

In the phrase How's it going $# this would how three matches: How's, it, going but wouldn't match the group of symbols.

There are lots of tutorials and even some games out there but you can start with regexone to understand it better by trying some out.

Answer 2

With help of re.compile('\W') we can remove special characters from the string.

Example :

str = 'how many $ amount spend for Car??'
pattern = re.compile('\W')
x = re.sub(pattern, ' ', str)
print(x)

Result:

how many amount spend for Car

Note: Special charter "$" and "?" are removed from the string.