Parallel.ForEach Behavior

Question

I understand that with a Parallel.ForEach loop each thread may be executing a different part of the loop at any given time. However, does each thread execute the code in the loop sequentially? I was just reading "Parallel Loops" on MSDN and it states:

Sometimes, two steps take place in the opposite order than they would if the loop were sequential. The only guarantee is that all of the loop's iterations will have run by the time the loop finishes.

Say I have the following:

IEnumerable<MyObject> myEnumerable = ...
Parallel.ForEach(myEnumerable, obj =>
{
     A();
     B();
     C();
});

I know that thread 1 may be doing A() while thread 2 might be doing C(), but will each thread execute the code sequentially in the loop. Does Thread 1 do A() B() C() or could it possibly do B(), C(), A()?

Answer 1

An iteration is a single execution of the whole loop body, exactly as it is expressed in your code.

The iterations can (and most probably will) start, run and finish in any order (which will depend on implementation details and runtime data, like what workers become available and when). But still, each individual iteration will be executed just as if the loop was sequential (like a plain foreach) instead of parallel.

In other words, it couldn't possibly do B() then C() then A() for any one item. It will always do A() then B() then C() for each item. You simply can't know which item will be processed in what order.