Have been getting many OverQuotaExceptions using Mail API in App Engine in the last few days, despite being way below quota according to the console

Question

In the last few days we are getting many of the following error in our App Engine logs:

com.google.apphosting.api.ApiProxy$OverQuotaException: The API call mail.Send() required more quota than is available.

This is despite being very much under quota based on what I can see in the Google Cloud console. The content of our emails that we send out has remained the same, and there have been no significant increases in our traffic recently. Note that not all Mail API calls are failing, many are still successful.

Is this a known issue with the App Engine Mail API right now? There have been no changes on our end that justify this, so I have to imagine this problem in on Google's end.

New, September 19: It seems that since this issue happened and was resolved, a different issue has come up. Some of the emails that we send out appear to be successfully sent out from an API perspective, but they are not received by any of the recipients (and are not in Spam folders). I believe this to be a small percentage of total emails that we send out on a daily basis, but some of our users have certainly noticed..

Has anyone else noticed this?

Answer 1

If you go to https://console.cloud.google.com/iam-admin/quotas?project=(xxx) you can see that "Mail messages sent to non-admins per day" have been lowered to 120, inexplicably.

This sudden change without any warning (that I know of) is absolutely dramatic and blocks our entire online invoicing business.