ConcurrentQueue.Count extremely slow in .NET Core

Question

I'm using a ConcurrentQueue to enqueue items from an I/O bound task and dequeue them from another for processing. I stop adding items to the queue when it reaches a certain size, so the processing can catch up. To do this I check the ConcurrentQueue.Count property.

The problem is that the Count property doesn't seem to behave as it does in a List or other collections. It's extremely slow, and the larger the queue the slower it takes to read the Count property. With 20k items in a ConcurrentQueue almost all processor time is spent on the Count property.

Rough Example:

        while (reader.Read())
        {
            if(Queue.Count >= MaxQueueSize)
            {
                //Wait
            }
            //Do Stuff
        }

When running a performance profiler, all the time is spent on System.Collections.Concurrent.CDSCollectionETWBCLProvicer.ctor().

This only seems to occur on .NET Core 2, this does not occur in .NET 4.6.2

Is there any way around this in .Net Core?

Answer 1

Since source code of both frameworks is available nowadays, one can take a peek at source code for both Count versions. For full .NET it's here, and for .NET Core it's here. You can see that:

1. Both versions are non-trivial. It's not something like return _count.

2. .NET Core version is quite more complicated, and it includes slow paths where whole queue is freezed just to calculate stable count.

So it's not surprising that .NET Core version is generally slower, but main point is - both of them are non-trivial and using them in a tight loop is not a good idea. In addition to that - you have a race condition by checking that Count, because it might be changed by another thread immediately after you check it, so queue is not really guaranteed to contain less than max count in your version.

Instead, use built-in capacity bounded collections, such as BlockingCollection:

var x = new BlockingCollection<string>(new ConcurrentQueue<string>(), MaxQueueSize);