Python Azure WebJob Import Error - Cannot import Python Extension Modules

Question

I am trying to run a Python script using an Azure WebJob, and it works fine when I want to make requests and generate CSVs. I've successfully zipped all the packages that I need along with my script and have uploaded them to the Azure App_Data directory.

However, I also need to be able to connect to an SFTP site, and some of the packages required contain Python extension modules. When I run the script locally. it works fine. However, when I run on Azure, I get a message saying "ImportError: cannot import name _bcrypt"

Here is my script:

import sys, os
sys.path.append(os.path.join(os.getcwd(), "site-packages"))

import pysftp
import paramiko

hostname = 'host'
username='user'
password='pass'
port='port'

source = 'D:\\Home\\PunchData.csv'
destination = 'PunchData_Success.csv'

client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(hostname=hostname,port=port,username=username,password=password)

ftp_client=client.open_sftp()

ftp_client.chdir('uploads')

ftp_client.put(source,destination)

ftp_client.close()

Here is the full error message I receive:

[12/15/2018 00:36:26 > 00ceeb: SYS INFO] Status changed to Initializing
[12/15/2018 00:36:28 > 00ceeb: SYS INFO] Job directory change detected: Job file 'enum\LICENSE' exists in source directory but not in working directory.
[12/15/2018 00:37:03 > 00ceeb: SYS INFO] Run script 'run.py' with script host - 'PythonScriptHost'
[12/15/2018 00:37:03 > 00ceeb: SYS INFO] Status changed to Running
[12/15/2018 00:37:06 > 00ceeb: ERR ] Traceback (most recent call last):
[12/15/2018 00:37:06 > 00ceeb: ERR ]   File "run.py", line 1, in <module>
[12/15/2018 00:37:06 > 00ceeb: ERR ]     import pysftp
[12/15/2018 00:37:06 > 00ceeb: ERR ]   File "D:\local\Temp\jobs\triggered\FTPTest\5zbuguvp.pts\pysftp\__init__.py", line 12, in <module>
[12/15/2018 00:37:06 > 00ceeb: ERR ]     import paramiko
[12/15/2018 00:37:06 > 00ceeb: ERR ]   File "D:\local\Temp\jobs\triggered\FTPTest\5zbuguvp.pts\paramiko\__init__.py", line 22, in <module>
[12/15/2018 00:37:06 > 00ceeb: ERR ]     from paramiko.transport import SecurityOptions, Transport
[12/15/2018 00:37:06 > 00ceeb: ERR ]   File "D:\local\Temp\jobs\triggered\FTPTest\5zbuguvp.pts\paramiko\transport.py", line 90, in <module>
[12/15/2018 00:37:06 > 00ceeb: ERR ]     from paramiko.ed25519key import Ed25519Key
[12/15/2018 00:37:06 > 00ceeb: ERR ]   File "D:\local\Temp\jobs\triggered\FTPTest\5zbuguvp.pts\paramiko\ed25519key.py", line 17, in <module>
[12/15/2018 00:37:06 > 00ceeb: ERR ]     import bcrypt
[12/15/2018 00:37:06 > 00ceeb: ERR ]   File "D:\local\Temp\jobs\triggered\FTPTest\5zbuguvp.pts\bcrypt\__init__.py", line 25, in <module>
[12/15/2018 00:37:06 > 00ceeb: ERR ]     from . import _bcrypt
[12/15/2018 00:37:06 > 00ceeb: ERR ] ImportError: cannot import name _bcrypt
[12/15/2018 00:37:06 > 00ceeb: SYS INFO] Status changed to Failed
[12/15/2018 00:37:06 > 00ceeb: SYS ERR ] Job failed due to exit code 1

It seems to fail any time a Python extension module is imported. I'm using a 32bit Python 2.7 environment on a Windows machine. I've read that for Azure functions, you need to utilize a Python wheel to replace Python extension modules, but I'm not sure how I would use a wheel for an Azure WebJob. Uploading the wheel to the WebJob directory doesn't solve the issue.

Any help would be much appreciated!

Answer 1

I figured out the solution.

Instead of uploading the Python packages in a zip file directly with the Python script into the WebJob, you can install the packages with pip in the WebApp. To do so, I first added a Handler Mapping in Application Settings to specify my Python version:

I then opened the Kudu console

Once I had the Kudu console open, I went to Site Extensions and made sure I had the version of Python installed that I wanted:

I then went to Debug Console > CMD to open the command prompt, and installed all my required packages using pip:

Once I had all the required packages installed, I was able to remove them from the WebJob directory. The WebJob script is able to use the packages that were just installed using pip. Once I ran it, I no longer had any issue with Azure being able to import Python extension modules.

Hopefully this helps someone out there who runs into a similar issue.