Error "ModuleNotFoundError: No module named 'psycopg2'" when deploying Django app to Google App Engine

Question

I'm working on a Django application and using Fabric for deployment.
When I deployed the apps to staging in Google App Engine using fab testing, I got this error:

Updating service [staging] (this may take several minutes)...failed.                                
ERROR: (gcloud.app.deploy) Error Response: [9] An internal error occurred while processing task /app-engine-flex/flex_await_healthy/flex_await_healthy>2022-01-01T09:48:30.226Z15496.fj.0: Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/env/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/env/lib/python3.6/site-packages/django/core/management/__init__.py", line 347, in execute
    django.setup()
  File "/env/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/env/lib/python3.6/site-packages/django/apps/registry.py", line 89, in populate
    app_config = AppConfig.create(entry)
  File "/env/lib/python3.6/site-packages/django/apps/config.py", line 116, in create
    mod = import_module(mod_path)
  File "/opt/python3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/env/lib/python3.6/site-packages/django/contrib/postgres/apps.py", line 8, in <module>
    from .signals import register_type_handlers
  File "/env/lib/python3.6/site-packages/django/contrib/postgres/signals.py", line 3, in <module>
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

I'm sure the psycopg2 has been installed successfully earlier. Here's the list installed dependencies after checking by pip list:

...
platformdirs                      2.4.0
prompt-toolkit                    3.0.24
protobuf                          3.19.1
psutil                            5.5.1
psycopg2                          2.8.6
pyasn1                            0.4.8
pyasn1-modules                    0.2.8
pycairo                           1.16.2
...

Anyone can help? Thanks!

Answer 1

Thank you guys for your prompt support. I've managed to resolve it. Basically during coding I've accidentally commented the psycopg2 line in the requirements.txt. Because earlier all dependencies were installed successfully and everything is running on dev mode properly so I just kept that line excluded out.
But when deploying with Fabric, it will execute the gcloud command (gcloud app deploy) and here's the thing: GCloud will re-run everything (including installing all dependencies all over again). And the psycopg2 wasn't installed (because it's commented out) during that GCloud runtime. That's why GCloud couldn't find it for further executions.
So I've put it back and everything works like a charm. What a stupid move of mine, but it's fantastic to understand how GCloud works on its way

Answer 2

Just add in your requirement.txt to psycopg2=2.9.3 or run the below command manually to your virtual environment pip install psycopg2