How to run a Bash script that calls a Python script from any location?

Question

I have a Python script, say myscript.py, that uses relative module imports, ie from .. import module1, where my project layout is as follows:

project
 + outer_module
   - __init__.py
   - module1.py
   + inner_module
     - __init__.py
     - myscript.py
     - myscript.sh

And I have a Bash script, say myscript.sh, which is a wrapper for my python script, shown below:

#!/bin/bash
python -m outer_module.inner_module.myscript $@

This works to execute myscript.py and forwards the arguments to my script as desired, but it only works when I call ./outer_module/inner_module/myscript.sh from within the project directory shown above.

How can I make this script work from anywhere? For example, how can I make this work for a call like bash /root/to/my/project/outer_module/inner_module/myscript.sh?

Here are my attempts:

When executing myscript.sh from anywhere else, I get the error: No module named outer_module.inner_module. Then I tried another approach to execute the bash script from anywhere, by replacing myscript.sh with:

#!/bin/bash
scriptdir=`dirname "$BASH_SOURCE"`
python $scriptdir/myscript.py $@

When I execute myscript.sh as shown above, I get the following:

Traceback (most recent call last):
  File "./inner_module/myscript.py", line 10, in <module>
    from .. import module1
ValueError: Attempted relative import in non-package

Which is due to the relative import on the first line in myscript.py, as mentioned earlier, which is from .. import module1.

Answer 1

As the error message says:

ValueError: Attempted relative import in non-package

The solution to this is to create a package, and have your script execute with that package in its path.

You already have a package, as you have the __init__.py files in those directories; but you only have that package in your path when you are calling it from the project directory, as you mentioned; that's because . is in your Python path by default.

To fix this, just add the project directory to your Python path, and then invoke it with python -m outer_module.inner_module.myscript:

#!/bin/bash
export PYTHONPATH=$PYTHONPATH:$(dirname "$BASH_SOURCE")/../..
python -m outer_module.inner_module.myscript $@

Answer 2

You need to include the path to the outer module's parent directory in the PYTHONPATH environmental variable, then you can use the same command you used in the first script from anywhere.

The PYTHONPATH is where python searches for any modules you try to import:

#!/bin/bash
export PYTHONPATH=$PYTHONPATH:PATH/TO/MODULE/PARENTDIR
python -m outer_module.inner_module.myscript $@