dir function not showing all package contents

Question

I'm trying to get a list of all modules inside a python package. This is my directory structure:

foo/
   __init__.py
   bar.py
   dumb.py
   dog.py

I'm trying to list all modules like this:

import foo

dir(foo)  
# ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']

As you can see the dir function doesn't list the modules of the package. But if I do something like this look what happens:

import foo
import foo.bar

dir(foo)  
# ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'bar']

Can someone explain what just happened? There's a way I can list all modules names without importing them?

Answer 1

Python's import mechanism isn't a directory listing, when you import foo only foo is imported along with a file named __init__.py (if it exists in that directory).

When a module is imported its code is executed and stored within sys.modules, and a use of __init__.py is that you use it to from . import bar thus allowing import foo to import whatever it is from the directory that you want.

When you ran your second line import foo.bar thus you told the interpreter to import bar (be it a file named bar.py or a directory named bar (with or without an __init__.py)) and that this bar module is a child of the foo module (the . which connects them).

p.s. this is a small abstraction on the import system, but I hope this answers your question.

EDIT: To get your desired results, you can use the imported modules metadata to inspect the folder imported.

waving my hands over the internals, here is a snippet

[pyimport]
$: tree
.
├── mod1
│   ├── __init__.py
│   └── inner.py
└── mod2
    └── inner.py

[pyimport]
$: py --no-banner

In [1]: import mod1

In [2]: import mod2

In [3]: mod1.__file__
Out[3]: '/home/yhoffman/Sandbox/pyfiles/pyimport/mod1/__init__.py'

In [4]: mod1.__path__
Out[4]: ['/home/yhoffman/Sandbox/pyfiles/pyimport/mod1']

In [5]: mod2.__file__

In [6]: mod2.__path__
Out[6]: _NamespacePath(['/home/yhoffman/Sandbox/pyfiles/pyimport/mod2'])

In [7]: import os

In [8]: os.listdir(os.path.dirname(mod1.__file__))
Out[8]: ['__init__.py', 'inner.py', '__pycache__']

In [9]: os.listdir(mod1.__path__[0])
Out[9]: ['__init__.py', 'inner.py', '__pycache__']

In [11]: os.listdir(mod2.__path__._path[0])
Out[11]: ['inner.py']