Find which Python package provides a specific import module

Question

Without getting confused, there are tons of questions about installing Python packages, how to import the resulting modules, and listing what packages are available. But there doesn't seem to be the equivalent of a --what-provides option for pip, if you don't have a pip-style requirements.txt file or a Pipenv Pipfile. This question is similar to a previous question, but asks for the parent package, and not additional metadata. That said, these other questions did not get a lot of attention or many accepted answers - eg. How do you find python package metadata information given a module. So forging ahead...

By way of example, there are two packages (to name a few) that will install a module called serial - namely pyserial and serial. So assuming that one of the packages was installed, we might find it by using pip list:

python3 -m pip list | grep serial

However, the problem comes in if the name of the package does not match the name of the module, or if you just want to find out what package to install, working on a legacy server or development machine.

You can check the path of the imported module - which can give you a clue. But continuing the example...

>>> import serial
>>> print(serial.__file__)
/usr/lib/python3.6/site-packages/serial/__init__.py

It is in a serial directory, but only pyserial is in fact installed, not serial:

> python3 -m pip list | grep serial
pyserial                 3.4

The closest I can come is to generate a requirements.txt via pipreqs ./ which may fail on a dependent child file (as it does with me), or to reverse check dependencies via Pipenv (which brings a whole set of new issues along to get it all setup):

> pipenv graph --reverse
cymysql==0.9.15
ftptool==0.7.1
netifaces==0.10.9
pip==20.2.2
PyQt5-sip==12.8.1
    - PyQt5==5.15.0 [requires: PyQt5-sip>=12.8,<13]
setuptools==50.3.0
wheel==0.35.1

Does anyone know of a command that I have missed for a simple solution to finding what pip package provides a particular module?

Answer 1

Use the packages_distributions() function from importlib.metadata (or importlib-metadata). So for example, in your case where serial is the name of the "import package":

import importlib.metadata  # or: `import importlib_metadata`

importlib.metadata.packages_distributions()['serial']

This should return a list containing pyserial, which is the name of the "distribution package" (the name that should be used to pip-install).

References

• https://importlib-metadata.readthedocs.io/en/stable/using.html#package-distributions
• https://github.com/python/importlib_metadata/pull/287/files

---

For older Python versions and/or older versions of importlib-metadata...

I believe something like the following should work:

#!/usr/bin/env python3

import importlib.util
import pathlib

import importlib_metadata

def get_distribution(file_name):
    result = None
    for distribution in importlib_metadata.distributions():
        try:
            relative = (
                pathlib.Path(file_name)
                .relative_to(distribution.locate_file(''))
            )
        except ValueError:
            pass
        else:
            if distribution.files and relative in distribution.files:
                result = distribution
                break
    return result

def alpha():
    file_name = importlib.util.find_spec('serial').origin
    distribution = get_distribution(file_name)
    print("alpha", distribution.metadata['Name'])

def bravo():
    import serial
    file_name = serial.__file__
    distribution = get_distribution(file_name)
    print("bravo", distribution.metadata['Name'])

if __name__ == '__main__':
    alpha()
    bravo()

This is just an example of code showing how to get the metadata of the installed project a specific module belongs to.

The important bit is the get_distribution function, it takes a file name as an argument. It could be the file name of a module or package data. If that file name belongs to a project installed in the environment (via pip install for example) then the importlib.metadata.Distribution object is returned.