I have an API endpoint on AWS API Gateway with AWS Lambda (Python & Flask) to store some data from a JSON file. e.g) curl -X POST http://www.xxx.yyy/store -d @zzz.json
However, when I tried executing the API with a bigger JSON file, I encountered a timeout error. Through my investigation, the maximum timeout setting for Lambda is 300 seconds, and API Gateway is 29 seconds. The maximum timeout for Lambda 300 sec sounds fine, but 29 seconds sounds too short. What kind of things could be a solution? The JSON data can be split by id, but it needs to be sent as one file.
EDIT: Sure I can't change the number. Any suggestion to solve this problem using another technology/system design pattern? I can't change the input, though.
EDIT 2: Currently, the Lambda function has validation based on JSON scheme, parse into models, and save into database. Any suggestions?
Update - June 4, 2024
AWS has removed the 29 second API Gateway limit:
Amazon API Gateway integration timeout limit increase beyond 29 seconds
Is there anyway you can update your Lambda function to hand off to another process?
By decoupling you could for example do the following:
API Gateway -> Lambda (Perform any mandatory action, then store in S3 as a blob) -> S3 -> Another Lambda to process.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With