Error: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /var/task/cryptography/hazmat/bindings/_rust.abi3.so)

Question

I need a 'for dummies' answer to this question that I know has been asked before.

We're using the Serverless framework for an AWS-hosted application. Runtime=python3.8 Got a nice big yml file that includes 16 functions, 2 of which include layers for Cryptography and for PyNaCl, which we bring in from here - https://github.com/keithrozario/Klayers and have used successfully for quite a while.

Last week, I needed to update a different function, which meant re-testing, which meant finding there's a newer version of the cryptography layer, so I updated it to have Cyptography v.39. Now I have a function that fails with the error, /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /var/task/cryptography/hazmat/bindings/_rust.abi3.so) This function hasn't been used since 07/2022, at which time it was just fine. Apparently it's also been that long since we redeployed from Serverless.

Attempts to fix:

• I reverted to the previous Cryptography layer; no difference
• I found an SO answer with this link https://aws.amazon.com/premiumsupport/knowledge-center/lambda-python-package-compatible/, followed that guide to change my local install and also to make my own layer and attach it in the console; no difference
• Another SO answer led me here - https://github.com/pyca/cryptography/issues/6390, which then goes to https://github.com/pyca/cryptography/issues/6391, which also didn't help
• Today, I found this link https://repost.aws/questions/QU85KE-2hPQ4KDQyByKV_WIw/creating-a-lambda-zip-package-that-runs-python-package-cryptography and the OP says they had to make all of their lambdas from x86_64 to arm64, even if those functions weren't using Cryptography. That's seems insane, and besides.... how??

This question Lambda function failing with /lib64/libc.so.6: version `GLIBC_2.18' not found includes the advice to move from Cryptography v.39 all the way back to v.3.4.7 (from 03/2021), which seems like bad advice. Surely the 14 updates between those 2 versions include some important changes.

I'm at a loss. I feel like I'm just running in circles, and meanwhile can't make progress on the actual function I'm trying to update because this is such a block.

Answer 1

I'm using Serverless and https://www.npmjs.com/package/serverless-python-requirements to bundle a Python 3.9 function, and was hitting the same error. I added a couple of pip args to the serverless plugin to specify the target system, which got me past the issue: pipCmdExtraArgs: ['--platform manylinux2014_x86_64', '--only-binary=:all:'] Here is the full config I'm using for the plugin (from my serverless.ts):

pythonRequirements: {
      dockerizePip: false,
      usePoetry: false,
      layer: true,
      useDownloadCache: false,
      useStaticCache: false,
      pipCmdExtraArgs: ['--platform manylinux2014_x86_64', '--only-binary=:all:'],
      slim: true
    }

which results in a pip install command that looks like this (just in case you are calling pip install manually in your pipeline): python3.9 -m pip install --platform manylinux2014_x86_64 --only-binary=:all: -t /someOutputFolder -r requirements.txt

Update: Another option I've found is to use a docker container as the lambda container, and install the python dependencies in the dockerfile, as shown in these docs: https://docs.aws.amazon.com/lambda/latest/dg/images-create.html Serverless makes it really easy to deploy the container image: https://www.serverless.com/blog/container-support-for-lambda