AWS Lambda -- Unable to import srsly.ujson.ujson for SpaCy

Question

I am trying to add SpaCy as a dependency to my Python Lambda. I am doing this by installing SpaCy as a standalone dependency inside a directory named dependencies using pip3 install spacy --no-deps -t . This is because I can't load the entire Spacy dependency inside the \tmp directory of my Lambda.

I am able to successfully upload the folder to s3 and download it during the Lambda invocation. When I try to import spacy, I get this error: [ERROR] Runtime.ImportModuleError: Unable to import module : No module named 'srsly.ujson.ujson'.

I manually installed srsly inside dependencies\ and I have all the files that are listed as per this link. This was referenced by this link. One of the responses says, "it seems like Python can't load it, because it's not compiled?". How would I compile a dependency which has a .c file in it?

One other question which I found on SO is this question, but I have already manually installed srsly. How to I import the module? Thanks.

---

I manually check in my code for the presence of ujson before importing spacy like this:

if os.path.exists('/tmp/dependencies/srsly/ujson/ujson.c'):
    print('ujson exists')

and the print statement gets printed.

Answer 1

For me pip uninstalling and installing srsly again worked fine.. sometimes its just the compatibility issue with your python version so make sure correct python/srsly versions are present