What is "where" argument for in setuptools.find_packages?

Question

working on a python project, I tried to separate source code and unit tests; here is the project structure:

MyProject/
    MANIFEST.in
    README.md
    setup.py
    source/
        __init.py__
        my_project/
            __init.py__
            some_module.py
    test/
        __init.py__
        my_project/
            __init.py__
            test_some_module.py

And here is the setup.py file:

from setuptools import setup, find_packages

setup(
    name='my_project',
    packages=find_packages(where='./source'),
    description='My project to be packaged',
    version='1.0.0',
    author='me'
    install_requires=[
        'fastnumbers~=2.0.1',
        'numpy~=1.14.1',
        'pandas~=0.22.0'
    ],
    extras_require={
        'dev': ['check-manifest'],
        'test': [
            'mock',
            'PyHamcrest',
            'pytest',
            'pytest-cov'
        ],
    }
)

Then, when I run command python3 setup.py sdist, it fails with the following output:

running sdist
running egg_info
writing my_project.egg-info/PKG-INFO
writing requirements to my_project.egg-info/requires.txt
writing dependency_links to my_project.egg-info/dependency_links.txt
writing top-level names to my_project.egg-info/top_level.txt
error: package directory 'my_project' does not exist

The resulting top_level.txt file looks fine:

 my_project

but it looks like the setuptools is not starting from source folder to find modules to be packaged.

1. Do I have to move setup.py and MANIFEST.in files into source folder?
2. But then, what is this where argument for in setuptools.find_packages function?

Answer 1

You are one step away from a working solution. Add

package_dir={
    '': 'source',
},

to the setup() arguments:

setup(
    ...,
    packages=find_packages(where='source'),
    package_dir={
        '': 'source',
    },
    ...
)

More info on packages remapping can be found in Listing whole packages section.

However, it looks like you made the source dir to a python package by placing an __init__.py in it. Was that intentional? Do you have import statements like

import source.my_project
from source.my_project.my_module import stuff

or similar, using source as package name? Then beware that the imports will fail once you install your built package because source is omitted when including sources on build. I see two ways:

1. either remove source/__init__.py, use package_dir like described above to make my_project to the top level package, omit source in imports (should you get any errors, just remove myproject-1.0.0.egg_info dir and recreate it with python setup.py egg_info), or
2. use source as top level package: don't use package_dir, look for packages in the project root dir (packages=find_packages() without explicitly stating where).