Cannot connect to SQS from EC2 instance due to time out

Question

The error I see is: Unable to execute http request : Connect to sqs.us-east-1.amazonaws.com : 443

org.apache.http.conn.ConnectTimeoutException: Connect to sqs.us-east-1.amazonaws.com:443 [sqs.us-east-1.amazonaws.com/54.239.27.172] failed: connect timed out
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
        at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
        at com.amazonaws.http.conn.$Proxy54.connect(Unknown Source)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1181)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
        ... 21 common frames omitted

The outbound rules for my security group are: All traffic 10.0.0.0/8

I cannot change the outbound rules to allow all internet - that is a limitation.

Any idea how we can do this?

Answer 1

Amazon SQS is an Internet-based service. To connect to the Amazon SQS Endpoint (sqs.us-east-1.amazonaws.com), the Amazon EC2 instance requires access to the Internet.

Your Amazon EC2 instance is in a private subnet, which means it does not have direct access to the Internet. Therefore, you would need:

• Either a NAT Instance or a NAT Gateway in the Public Subnet of the VPC
• A Route Table associated with the Private Subnet that routes Internet-bound traffic to the NAT Instance or NAT Gateway

Answer 2

As John stated, AWS APIs are public endpoints (Exceptions are VPC Endpoints, which are VPC-private endpoints, currently only available for S3 and DynamoDB). From a private subnet you need a NAT to access those public endpoints.

However, in order to limit access from your instances to only AWS services, you need to configure the security group's egress rules.

Amazon publishes IP-Ranges for its endpoints and also allows you to subscribe to change notifications:

Whenever there is a change to the AWS IP address ranges, we send notifications to subscribers of the AmazonIpSpaceChanged topic. The payload contains information in the following format:

{
  "create-time":"yyyy-mm-ddThh:mm:ss+00:00",
  "synctoken":"0123456789",
  "md5":"6a45316e8bc9463c9e926d5d37836d33",
  "url":"https://ip-ranges.amazonaws.com/ip-ranges.json"
}

What you get from https://ip-ranges.amazonaws.com/ip-ranges.json is a json describing AWS services and their IP-ranges.

{
  "syncToken": "0123456789",
  "createDate": "yyyy-mm-dd-hh-mm-ss",
  "prefixes": [
    {
      "ip_prefix": "cidr",
      "region": "region",
      "service": "subset"
    }
  ],
  "ipv6_prefixes": [
    {
      "ipv6_prefix": "cidr",
      "region": "region",
      "service": "subset"
    }
  ]  
}

In order to restrict egress traffic, add a rule to your SG following these instructions from the AWS documentation:

To allow an instance to access only AWS services, create a security group with rules that allow outbound traffic to the CIDR blocks in the AMAZON list, minus the CIDR blocks that are also in the EC2 list.

For current details and IP-range json format, see section "Implementing Egress Control" in Amazon's general documentation