AWS Lambda Python 3.10 - No module named '_cffi_backend'

Question

I am running a Lambda in AWS using Python 3.10 and trying to run some code that builds ssh key-pairs. I get the following error when running my function, can anyone identify how to resolve this?

Error

strap.py", line 39, in _get_handler
m = importlib.import_module(modname.replace("/", "."))
File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/var/task/index.py", line 4, in <module>
import libraries.ssh as ssh
File "/var/task/libraries/ssh.py", line 3, in <module>
from cryptography.hazmat.primitives import serialization as crypto_serialization
File "/opt/python/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/__init__.py", line 7, in <module>
from cryptography.hazmat.primitives._serialization import (
File "/opt/python/lib/python3.10/site-packages/cryptography/hazmat/primitives/_serialization.py", line 11, in <module>
from cryptography.hazmat.primitives.hashes import HashAlgorithm
File "/opt/python/lib/python3.10/site-packages/cryptography/hazmat/primitives/hashes.py", line 10, in <module>
from cryptography.hazmat.bindings._rust import openssl as rust_openssl
pyo3_runtime.PanicException: Python API call failed
ModuleNotFoundError: No module named '_cffi_backend'
thread '<unnamed>' panicked at 'Python API call failed', /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/err/mod.rs:790:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
File "/var/runtime/bootstrap.py", line 60, in <module>
main()
File "/var/runtime/bootstrap.py", line 57, in main
awslambdaricmain.main([os.environ["LAMBDA_TASK_ROOT"], os.environ["_HANDLER"]])
File "/var/runtime/awslambdaric/__main__.py", line 21, in main
bootstrap.run(app_root, handler, lambda_runtime_api_addr)
File "/var/runtime/awslambdaric/bootstrap.py", line 389, in run
request_handler = _get_handler(handler)
File "/var/runtime/awslambdaric/bootstrap.py", line 39, in _get_handler
m = importlib.import_module(modname.replace("/", "."))
File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/var/task/index.py", line 4, in <module>
import libraries.ssh as ssh
File "/var/task/libraries/ssh.py", line 3, in <module>
from cryptography.hazmat.primitives import serialization as crypto_serialization
File "/opt/python/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/__init__.py", line 7, in <module>
from cryptography.hazmat.primitives._serialization import (
File "/opt/python/lib/python3.10/site-packages/cryptography/hazmat/primitives/_serialization.py", line 11, in <module>
from cryptography.hazmat.primitives.hashes import HashAlgorithm
File "/opt/python/lib/python3.10/site-packages/cryptography/hazmat/primitives/hashes.py", line 10, in <module>
from cryptography.hazmat.bindings._rust import openssl as rust_openssl
pyo3_runtime.PanicException: Python API call failed
START RequestId: 96dc8ff0-12e4-4ba3-8b7a-f87a1aff3419 Version: $LATEST
RequestId: 96dc8ff0-12e4-4ba3-8b7a-f87a1aff3419 Error: Runtime exited with error: exit status 1
Runtime.ExitError

Python Code Snippet

from cryptography.hazmat.primitives import serialization as crypto_serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.backends import default_backend as crypto_default_backend
import paramiko
import io
import boto3
import json

Pip command installing dependencies

"pip3 install --platform manylinux2014_x86_64 --only-binary=:all: -t ${path.module}/layer/python/lib/python3.10/site-packages -r ${path.module}/layer/requirements.txt"

requirements.txt file

paramiko

Supporting Info

• I am running the code that AWS publishes. That is the full piece of code that I am using.
• I moved the dependencies to a lambda layer
• I am building the zip via a github action runner (Ubuntu)

Ive tried a few variations of addition additional libraries to requirements.txt such as cffi with no success.

Answer 1

I got this to work but only under some specific conditions.

First, in my requirements.txt file I included the following:

cffi 
parami

Second, when creating the Layer for my lambda to use, I included some flags:

pip3 install --platform manylinux2014_x86_64 --only-binary=:all: -t ${path.module}/my-lambda-layer-dir/python/ -r ${path.module}/my-lambda-layer-dir/requirements.txt

Finally, I deployed this using Github actions, which are using the default ubuntu-latest runner.

I explicitly set my Github action to use python 3.10:

  - uses: actions/setup-python@v4 
    with:
      python-version: '3.10' 

Getting it to work with Github actions using python 10 was just a fluke because my Lambda is running python 3.9. I suspect that there is no released version of the required packages for python 3.10 so its just picking up the latest at version 3.9.

I wrapped this all in Terraform to deploy it. I could get it to work when building / applying was terraform via Github actions; it would not work when deploying via Terraform Cloud (my guess is some difference in the OS).