Getting around SNS API call limits to publish

Question

I need to publish a unique message to potentially thousands of device endpoints simultaneously.

The message is unique so I cant group the endpoints in to topics...

Although I cant find any documentation, it seems that SNS limits to only 10 concurrent API publish requests.

More than 10 concurrent requests returns

RequestError: send request failed
caused by: Post https://sns.us-east-1.amazonaws.com/: dial tcp 54.239.24.226:443: i/o timeout  

And then seems to block my IP from further requests for a short time...

I was planing to have the whole app backend to be "serverless" which would mean that there would be a scheduled task in Lambda to make the calls to SNS publish...

1000 SNS publish requests / 10 concurrent = 100 batches...This would mean it would take 100 * x seconds to process all the messages which would reach the API gateway and Lambda timeout limits (and would also add to the costs)

Is there a good way around these limits. A increase in allowed concurrent API calls would be nice...

Answer 1

Amazon SNS does not enforce a rate limit for Publish calls. Occasionally, SNS will throttle requests but the service will respond with HTTP 400 and an AWS SNS request ID.

The error message you posted looks like something upstream between you and the SNS endpoint is rate limiting your calls. Check if there is a proxy or firewall between you and the SNS endpoint, or talk to your network administrator.

Answer 2

You can request additional limit increases here:

https://console.aws.amazon.com/support/cases#/create?issueType=service-limit-increase&limitType=service-code-sns