Is ParallelOptions.MaxDegreeOfParallelism applied globally over multiple concurrent Parallel calls?

Question

Consider this code run on a CPU with 32 cores:

ParallelOptions po = new ParallelOptions();
po.MaxDegreeOfParallelism = 8;

Parallel.For(0, 4, po, (i) =>
   {
      Parallel.For(0, 4, po, (j) =>
         {
            WorkMethod(i, j);  // assume a long-running method
         });
   }
);

My question is what is the actual maximum possibly concurrency of WorkMethod(i, j)? Is it 4, 8, or 16?

Answer 1

ParallelOptions.MaxDegreeOfParallelism is not applied globally. If you have enough cores, and the scheduler sees fit you will get a multiplication of the nested MPD values with each For able to spin up that many tasks (if the workloads are unconstrained).

Consider this example, 3 tasks can start 3 more tasks. This is limited by the MDP option of 3.

int k = 0;
ParallelOptions po = new ParallelOptions();
po.MaxDegreeOfParallelism = 3;

Parallel.For(0, 10, po, (i) =>
{
   Parallel.For(0, 10, po, (j) =>
         {
            Interlocked.Increment(ref k);
            Console.WriteLine(k);
            Thread.Sleep(2000);               
            Interlocked.Decrement(ref k);
         });
   Thread.Sleep(2000);
});

Output

1
2
3
4
7
5
6
8
9
9
5
6
7
9
9
8
8
9
...

If MDP was global you would only get 3 I guess, since it's not you get 9s.