Lambda function package still large despite using a layer for dependencies

Question

I have a Python Lambda and since I started using AWS X-Ray the package size has ballooned from 445KB to 9.5MB.

To address this and speed up deployments of my code, I have packaged my requirements separately and added a layer to my template. The documentation suggests that this approach should work.

Packaging dependencies in a layer reduces the size of the deployment package that you upload when you modify your code.

pip install --target ../package/python -r requirements.txt

Resources:
  ...

  ProxyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Architectures:
        - x86_64
      CodeUri: proxy/
      Handler: app.lambda_handler
      Layers:
        - !Ref ProxyFunctionLibraries
      Role: !GetAtt ProxyFunctionRole.Arn
      Runtime: python3.8
      Tracing: Active

  ProxyFunctionLibraries:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: proxy-function-lib
      Description: Dependencies for the ProxyFunction.
      ContentUri: package/.
      CompatibleRuntimes:
        - python3.8

However, this doesn't seem to have prevented the Lambda from still packaging everything in the top layer, and every time I deploy the package is still 9.5MB. The new layer for some reason is 11MB in size, but that is only being deployed when a change is made.

How can I reduce the size of the Lambda function package?

Answer 1

Actually the solution here was quite simple, although not obvious to non-Lambda experts.

As described in the question, the first step was to build the package library.

pip install --target ../package/python -r requirements.txt

However, when building the Lambda using sam build -u the same 'requirements.txt' file is used and the required dependencies were again being installed, this time as part of the app.

So all I had to do was remove the requirements that I wish packaged in a separate layer and rebuild. It does mean that I have to maintain 2x 'requirements.txt' but that is entirely manageable.

I've opened an issue and hopefully AWS will update their documentation.