org.apache.http.ConnectionClosedException: Connection closed - PoolingNHttpClientConnectionManager

Question

We are getting the below error frequently in our production. The reason is, server closes the connection and the client is trying with half closed stale connection. we have eviction strategy which runs every 80 minutes periodically which will check for expired and idle connections and close them. still we are getting this errors. we are planning to reduce the eviction thread interval to 40 minutes. is there any other solution available to stop this errors?

We are using PoolingNHttpClientConnectionManager. idle period timeout is 60 seconds.httpasyncclient version is 4.1.1.

org.apache.http.ConnectionClosedException: Connection closed
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:341)
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:263)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:123)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:164)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:339)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:317)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:278)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:590)
        at java.lang.Thread.run(Thread.java:748)

Answer 1

There are two options:

1. Define a finite TTL (total time to live) limit for all connections at the connection manager construction time
2. Use a custom ConnectionKeepAliveStrategy implementation to enforce a relative expiration time for persistent connections.

Here is example of HttpAsyncClient configuration that enforces a total time to live limit of 2 minute and relative expiration time of 30 seconds

Please note this may not completely eliminate the issue as both endpoints, the client and the server, may close connections at any point of time without a prior handshake.

ConnectingIOReactor ioReactor = new DefaultConnectingIOReactor(IOReactorConfig.custom()
        .build());
PoolingNHttpClientConnectionManager cm = new PoolingNHttpClientConnectionManager(
        ioReactor,
        ManagedNHttpClientConnectionFactory.INSTANCE,
        RegistryBuilder.<SchemeIOSessionStrategy>create()
                .register("http", NoopIOSessionStrategy.INSTANCE)
                .register("https", SSLIOSessionStrategy.getSystemDefaultStrategy())
                .build(),
        DefaultSchemePortResolver.INSTANCE,
        SystemDefaultDnsResolver.INSTANCE,
        2,
        TimeUnit.MINUTES);
CloseableHttpAsyncClient client = HttpAsyncClients.custom()
        .setConnectionManager(cm)
        .setConnectionManagerShared(false)
        .setKeepAliveStrategy(new DefaultConnectionKeepAliveStrategy() {
            @Override
            public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
                long keepAliveDuration = super.getKeepAliveDuration(response, context);
                if (keepAliveDuration > 30000) {
                    keepAliveDuration = 30000;
                }
                return keepAliveDuration;
            }
        })
        .build();