Why is a ConcurrentDictionary faster than a Dictionary in benchmark?

Question

I have a really simple benchmark to measure and compare performance of Dictionary<string, int> and ConcurrentDictionary<string, int>:

[MemoryDiagnoser]
public class ExtremelySimpleDictionaryBenchmark
{
    private readonly List<string> _keys = new();
    private readonly Dictionary<string, int> _dict = new ();
    private readonly ConcurrentDictionary<string, int> _concurrentDict = new ();

    private const int DictionarySize = 300;
    private const string Alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";

    private static string Generate(int length, Random random)
    {
        var result = new char[length];
        for (var i = 0; i < length; i++)
            result[i] = Alphabet[random.Next(Alphabet.Length)];
        return new string(result);
    }
    
    [GlobalSetup]
    public void Setup()
    {
        var rnd = new Random();
        for (var i = 0; i < DictionarySize; i++)
        {
            var key = Generate(6, rnd);
            var count = rnd.Next(1000);
            _dict[key] = count;
            _concurrentDict[key] = count;
            _keys.Add(key);
        }
    }
    
    [Benchmark(Baseline = true)]
    public int Dictionary()
    {
        // variable res is used to avoid dead code elimination
        var res = 0;
        for (var index = 0; index < _keys.Count; index++)
            if(_dict.TryGetValue(_keys[index], out var value))
                res += value;
        return res;
    }

    [Benchmark]
    public int ConcurrentDictionary()
    {
        // variable res is used to avoid dead code elimination
        var res = 0;
        for (var index = 0; index < _keys.Count; index++)
            if(_concurrentDict.TryGetValue(_keys[index], out var value))
                res += value;
        return res;
    }
}

It produces a really weird results. ConcurrentDictionary.TryGetvalue beats Dictionary.TryGetvalue in terms of performance. I have the following results:

Method	Mean	Error	StdDev	Ratio	Allocated	Alloc Ratio
Dictionary	2.055 us	0.0230 us	0.0215 us	1.00	-	NA
ConcurrentDictionary	1.522 us	0.0172 us	0.0152 us	0.74	-	NA

The benchmark was ran on .NET Core 8, Windows 11. Processor is Intel Core i7-14650HX. How can such results be explained?

Answer 1

@KevinGosse ran the benchmark on .NET 6, .NET 7, .NET 8 and .NET 9, and posted these results on GitHub:

| Method               | Job                | Runtime            | Mean     | Error     | StdDev    | Ratio | Allocated | Alloc Ratio |
|--------------------- |------------------- |------------------- |---------:|----------:|----------:|------:|----------:|------------:|
| Dictionary           | .NET 6.0           | .NET 6.0           | 2.043 us | 0.0075 us | 0.0071 us |  1.00 |         - |          NA |
| ConcurrentDictionary | .NET 6.0           | .NET 6.0           | 3.263 us | 0.0186 us | 0.0174 us |  1.60 |         - |          NA |
|                      |                    |                    |          |           |           |       |           |             |
| Dictionary           | .NET 7.0           | .NET 7.0           | 2.017 us | 0.0036 us | 0.0034 us |  1.00 |         - |          NA |
| ConcurrentDictionary | .NET 7.0           | .NET 7.0           | 3.304 us | 0.0116 us | 0.0102 us |  1.64 |         - |          NA |
|                      |                    |                    |          |           |           |       |           |             |
| Dictionary           | .NET 8.0           | .NET 8.0           | 1.945 us | 0.0045 us | 0.0042 us |  1.00 |         - |          NA |
| ConcurrentDictionary | .NET 8.0           | .NET 8.0           | 1.298 us | 0.0012 us | 0.0011 us |  0.67 |         - |          NA |
|                      |                    |                    |          |           |           |       |           |             |
| Dictionary           | .NET 9.0           | .NET 9.0           | 1.914 us | 0.0056 us | 0.0052 us |  1.00 |         - |          NA |
| ConcurrentDictionary | .NET 9.0           | .NET 9.0           | 1.313 us | 0.0049 us | 0.0045 us |  0.69 |         - |          NA |
|                      |                    |                    |          |           |           |       |           |             |
| Dictionary           | .NET Framework 4.8 | .NET Framework 4.8 | 3.415 us | 0.0126 us | 0.0118 us |  1.00 |         - |          NA |
| ConcurrentDictionary | .NET Framework 4.8 | .NET Framework 4.8 | 3.799 us | 0.0203 us | 0.0190 us |  1.11 |         - |          NA |

Apparently the ConcurrentDictionary<K,V> got an impressive performance boost on .NET 8, that more than doubled its reading performance, surpassing the Dictionary<K,V> that got no improvement. My guess is that the boost is related to this pull request by Stephen Toub, submitted on Feb 2, 2023 and titled "Improve ConcurrentDictionary performance, in particular for strings".

Update: I also ran an improvised benchmark (not BenchmarkDotNet) to measure the TryGetValue performance for non-string keys, and got inconsistent results on the .NET Fiddle server and on my PC. Here are the .NET Fiddle results (.NET 9):

Keys count: 100

Key: String, Iterations: 50,000,000
Dictionary:           1,042 msec, Sum: 2,525,000,000
ConcurrentDictionary: 914 msec, Sum: 2,525,000,000 (0.88 ratio)

Key: Int32, Iterations: 100,000,000
Dictionary:           882 msec, Sum: 5,050,000,000
ConcurrentDictionary: 792 msec, Sum: 5,050,000,000 (0.90 ratio)

Key: Object, Iterations: 50,000,000
Dictionary:           1,060 msec, Sum: 2,525,000,000
ConcurrentDictionary: 840 msec, Sum: 2,525,000,000 (0.79 ratio)

And here are the results on my Windows 10, quad core AMD Athlon machine (.NET 9 release build):

Keys count: 100

Key: String, Iterations: 50,000,000
Dictionary:           2,907 msec, Sum: 2,525,000,000
ConcurrentDictionary: 2,205 msec, Sum: 2,525,000,000 (0.76 ratio)

Key: Int32, Iterations: 100,000,000
Dictionary:           2,159 msec, Sum: 5,050,000,000
ConcurrentDictionary: 1,180 msec, Sum: 5,050,000,000 (0.55 ratio)

Key: Object, Iterations: 50,000,000
Dictionary:           2,942 msec, Sum: 2,525,000,000
ConcurrentDictionary: 2,273 msec, Sum: 2,525,000,000 (0.78 ratio)

On my machine the ConcurrentDictionary<K,V> is significantly faster for int keys.