Google App Engine "Exceeded soft memory limit on "

Question

We have a long-time running service on Google App Engine(GAE). However, after a while, we began to get the error in the image. No changes to the code. I don't understand why we're having the problem we didn't have before.

2020-01-05 08:31:32.704 UTC-8 Exceeded soft memory limit of 2048 MB with 2068 MB after servicing 0 requests total. Consider setting a larger instance class in app.yaml.

2020-01-05 08:31:32.705 UTC-8 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.

2020-01-05 08:31:32.705 UTC-8 The warmup request failed. Please check your warmup handler implementation and make sure it's working correctly.

Edit: System Info;

We are running python. I have cronjobs.

Answer 1

Do you know, how much memory it consumes when you run it locally?

With Stackdriver Profiler which is in beta, you can find what's happening with your memory usage.

When there is the Exceeded soft private memory limit error, you can follow two alternatives:

1) You can upgrade your instance to an another with more memory

2) You can reduce the chunks of data you process in each request, process the XML file in smaller pieces and keep the smaller instance doing the work.

Here is a similar post which can help.