Azure QueueClient MaxConcurrentCalls Not Working

Question

I have been tasked to improve the performance of a worker role which processes the messages from the queue.

It uses queueclient.OnMessage model with MaxConcurrentCalls setting as 1.

During the processing there is thread.sleep for 5 minutes and that will hamper the overall performance i.e. if I have 10 messages in the queue it will take at least 45 minutes before it processes the 10th message

I thought if I change MaxConcurrentCalls to let's say 5 then it should process 5 messages in parallel and reduce the waiting time by 25 minutes but that's not working :(

Also I tried to use OnMessageAsync with MaxConcurrentCalls but no luck. below is the snippet I tried,

.OnMessageAsync( async (brokeredMessage) =>
{                
    bool shouldAbandon = false;
    try
    {
        logger.Debug("Rcvd:" + brokeredMessage.SequenceNumber);
        SomeTask(brokeredMessage);
        await brokeredMessage.CompleteAsync();
    }
    catch (Exception ex)
    {
        logger.Error(String.Format("An Error occured {0}", ex.ToString()));
        shouldAbandon = true;
    }
    if (shouldAbandon)
    {
        await brokeredMessage.AbandonAsync();
    }   
}, new OnMessageOptions { AutoComplete = false, MaxConcurrentCalls = 10 });

private void SomeTask(BrokeredMessage bm)
{            
    logger.Debug("id: " + bm.MessageId + "on thread: " + Thread.CurrentThread.ManagedThreadId);
    Thread.Sleep(30 * 1000);
    logger.Debug("seq: " + bm.SequenceNumber);
    logger.Debug("body: " + bm.GetBody<string>());
}

Last option I can think is to start a new Task on OnMessage event. There are couple of things I'll need to take care of (making sure completed tasks are removed from the collection/main thread and pass a object from BrokeredMessage because BrokeredMessage is disposed so I can't use on the Task thread) and I have sanity tested it as well but I'm not convinced that this is the best solution.

Answer 1

Replace SomeTask implementation with the following:

async Task SomeTask(BrokeredMessage bm)
{
     logger.Debug("id: " + bm.MessageId + "on thread: " + Thread.CurrentThread.ManagedThreadId);
     await Task.Delay(30 * 1000).ConfigureAwait(false);
     logger.Debug("seq: " + bm.SequenceNumber);
     logger.Debug("body: " + bm.GetBody<string>());
}

When Thread.Sleep is mixed with async on the same thread (in your case), it causes thread to 'stall' all the tasks. Since there's no UI work, prefer to use .ConfigureAwait(false) on async operation to allow scheduler to manage what thread to execute on.