Using multiple HttpClient objects in an ASP.NET Core application

Question

What would be the best application design when there are multiple clients (companies) working with ASP.NET Core application and a separate instance of HttpClient with the corresponding client certificate must be maintained in ASP.NET application for each of these companies?

First, some important facts about the application:

• It's an ASP.NET Core 3.1 application.
• Each company using this ASP.NET Core application needs first to upload its own client certificate during the registration. Certificate and corresponding password are stored in the database. Certificate is necessary for communication with some external web service which must be notified on behalf of the company that some action was performed by the company in the mentioned ASP.NET Core application. In other words, for some actions company performs in the ASP.NET Core application, some external web service must be notified with POST method and corresponding body and its response must be processed and returned.
• After registration, company can start sending requests to the mentioned ASP.NET Core application. This ASP.NET Core application should obtain corresponding company certificate from the database, create HttpClient object and add certificate to it, prepare request data and call external web service for notification. Then response from this external web service should be processed and returned as response to the original company request.
• Of course, there can be simultaneous requests coming from different companies, but there can also be simultaneous requests coming for the same company from different company offices - all requests from the same company use the same company client certificate, but request data (body) to notify external web service will always be different.

As far as I know, it's not the best practice to create new HttpClient object for every call to external web service. I have considered HttpClientFactory, but in my case each HttpClient object must contain corresponding client certificate for the company. I know ASP.NET Core supports named HttpClient to maintain separate HttpClient for each company, but AFAIK this named clients can only be created in the Startup class. This won't be sufficient because new company can be registered anytime while ASP.NET app is running, company can upload new certificate while app is running rendering existing named client invalid, etc.

I am considering maintaining a static list containing one HttpClient object for each company. When first request for the company is initiated, new HttpClient is created with the corresponding client certificate and added to this list. For all subsequent requests from the same company (or office) the corresponding HttpClient is obtained from the mentioned static list and reused. Of course, I would have to establish some locking on the company's HttpClient instance so that it does not break in case of simultaneous requests from the same company. One concern I have with this design is that there might be couple of hundred companies and this list of http clients can be quite long.

Do you have any other idea or proposal?

Answer 1

I am considering maintaining a static list containing one HttpClient object for each company.

At a high level I think single-instance-per-company is the right approach, but the devil's in the (implementation) details.

Of course, I would have to establish some locking..

Not if you're using a type that was built for this kind of usage. ConcurrentDictionary, and specifically its GetOrAdd method, will give you a keyed storage mechanism where items are created lazily and is thread-safe and lock-free.

One concern I have with this design is that there might be couple of hundred companies and this list of http clients can be quite long.

I don't know if there's a specific number where you need to start worrying (depends on the OS and hardware, I believe), but at least according to the article that made the HttpClient singleton advice famous, it's in the thousands, not hundreds.

"In the production scenario I had the number of sockets was averaging around 4000, and at peak would exceed 5000, effectively crushing the available resources on the server, which then caused services to fall over. After implementing the change, the sockets in use dropped from an average of more than 4000 to being consistently less than 400, and usually around 100."

Still, if this is a concern, one thing you could do to mitigate it is allow those cached HttpClient instances to expire occasionally. (This should also mitigate that other famous problem.) Unfortunately, ConcurrentDictionary doesn't provide that capability out of the box. MemoryCache does, but doesn't directly support the lazy GetOrAdd semantics like ConcurrentDictionary. For the best of both worlds, have a look at LazyCache.