How to fix intermittent Serverless Image Handler 500 errors after moving AWS accounts?

Question

We have transferred our website to a new AWS account. Which meant moving all the images into a new bucket and setting up Serverless Image Handler in CloudFormation. I think we have upgraded Serverless Image Handler from v5.2.0 > v6.0.0. When we now load the website we get 500 errors from an intermittent number of images (within 'img' tags). The erroring images seem to lower over time. I.e. if we have 10 images on the page broken today, it could be 5 tomorrow.

If you copy one of these errored images you get the following:

ERRORED IMAGE

curl 'https://img2.picle.io/eyJlZGl0cyI6eyJyb3RhdGUiOm51bGwsInJlc2l6ZSI6eyJ3aWR0aCI6NjAwLCJoZWlnaHQiOjQ1MCwiZml0IjoiY292ZXIifX0sImJ1Y2tldCI6InByb2QuaW1nMi5waWNsZS5pbyIsImtleSI6ImltYWdlc1wvWW5QcDFLMkpcLzI3ZTAxOWVlLTMwY2ItNDU0ZS05ODAxLWZhMzJlOTdkYTE1OSJ9' \
  -H 'authority: img2.picle.io' \
  -H 'accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8' \
  -H 'accept-language: en-US,en;q=0.9,nb;q=0.8,it;q=0.7,la;q=0.6' \
  -H 'cache-control: no-cache' \
  -H 'pragma: no-cache' \
  -H 'referer: https://picle.io/' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  -H 'sec-fetch-dest: image' \
  -H 'sec-fetch-mode: no-cors' \
  -H 'sec-fetch-site: same-site' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36' \
  --compressed

This returns:

{"message": "Internal server error"}

But if we load the same image URL directly within a browser we get a correct 200 image:

curl 'https://img2.picle.io/eyJlZGl0cyI6eyJyb3RhdGUiOm51bGwsInJlc2l6ZSI6eyJ3aWR0aCI6NjAwLCJoZWlnaHQiOjQ1MCwiZml0IjoiY292ZXIifX0sImJ1Y2tldCI6InByb2QuaW1nMi5waWNsZS5pbyIsImtleSI6ImltYWdlc1wvWW5QcDFLMkpcLzI3ZTAxOWVlLTMwY2ItNDU0ZS05ODAxLWZhMzJlOTdkYTE1OSJ9' \
  -H 'authority: img2.picle.io' \
  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'accept-language: en-US,en;q=0.9,nb;q=0.8,it;q=0.7,la;q=0.6' \
  -H 'cache-control: no-cache' \
  -H 'pragma: no-cache' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  -H 'sec-fetch-dest: document' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-site: none' \
  -H 'sec-fetch-user: ?1' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36' \
  --compressed

Note, after a while both the above will probably start working. I'm not sure if its an import issue, it could take a while to re-process all the images again.

Answer 1

I found the answer to this in this thread!:

https://github.com/aws-solutions/serverless-image-handler/issues/375#issuecomment-1227290911

tl;dr

Make sure your "concurrent Lambda executions quota" is large enough (for me was 10, increased to 1000), otherwise if you request multiple images at once, some of them will get dropped out and fail, and then CloudFront will also cache them for a few minutes.

So the fix is to request an increase of the concurrent Lambda executions quota here: https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html

Hope this fixes the issue for you as well.

P.S: btw I just noticed this Github thread is actually opened by you haha, so all credit to you! :D

---

Make sure to select Concurrent executions when requesting a quota increase.