Node MaxListenersExceededWarning when I use same elasticsearch instance in a for loop for consecutive requests

Question

I currently have an expressJs app using latest @elastic/elasticsearch v8.8.1 javascript library. Now I have a class called CacheService that has a class variable client = new Client({ node: 'elasticsearch url here' }); and a method where I query for an existing cache stored inside an index.

async getCachedData(key) {            
            const body  = await this.client.search({
                index: this.index,
                body: {
                    query: {
                        match: { _id: key }
                    }
                }
            });
    
            const hits = body.hits.hits;
            if (hits.length > 0) {
                const cacheData =  hits[0]._source.data;
                const timestamp = hits[0]._source.timestamp;
                return { data: cacheData, timestamp: timestamp };
            }
    }

Now, if I call this method in a for loop more than 11 times, I get the following exception:

MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 abort listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
    at _addListener (node:events:444:17)
    at EventEmitter.addListener (node:events:460:10)
    at addSignal (/usr/src/app/node_modules/undici/lib/api/abort-signal.js:35:19)
    at new RequestHandler (/usr/src/app/node_modules/undici/lib/api/api-request.js:68:5)
    at Pool.request (/usr/src/app/node_modules/undici/lib/api/api-request.js:170:25)
    at /usr/src/app/node_modules/undici/lib/api/api-request.js:163:15
    at new Promise (<anonymous>)
    at Pool.request (/usr/src/app/node_modules/undici/lib/api/api-request.js:162:12)
    at Connection.request (/usr/src/app/node_modules/@elastic/transport/lib/connection/UndiciConnection.js:143:41)
    at SniffingTransport.request (/usr/src/app/node_modules/@elastic/transport/lib/Transport.js:412:75)
    at Client.SearchApi [as search] (/usr/src/app/node_modules/@elastic/elasticsearch/lib/api/api/search.js:66:33)

There is something with querying ES using the same instance of the client more than 11 times.

The only potential solution I found is if I use a local variable in getCachedData like so let client = new Client({ node: elasticUrl }); and after await client.search I use client.close(), the warning disappears.

async getCachedData(key) {
            let client = new Client({ node: 'url here' });       
            const body  = await client.search({
                index: this.index,
                body: {
                    query: {
                        match: { _id: key }
                    }
                }
            });

            client.close();
    
            const hits = body.hits.hits;
            if (hits.length > 0) {
                const cacheData =  hits[0]._source.data;
                const timestamp = hits[0]._source.timestamp;
                return { data: cacheData, timestamp: timestamp };
            }
    }

Any idea what I'm doing wrong? I tried everything, this was the only solution, but I'm not sure about client.close() after every request as it adds overhead and potential can impact performance?

Thank you!

Answer 1

This is an issue with undici and here are more details about it.

https://github.com/elastic/elastic-transport-js/issues/63

If you just want to increase the max listeners on your client you can call

const max = 100; // set to whatever you want your max to be
client.diagnostic.setMaxListeners(max)

However, this will be a moving target depending on the size of items you want to loop over.

If you are able to rewrite this using msearch that may be better in the long run.