Check for duplicates in Blocking collection [duplicate]

Question

What is the best way to check if an item is existed in the BlockingCollection before trying to add a new one? Basically I do not want duplicates to be added to the BlockingCollection.

Answer 1

You will have to implement your own IProducerConsumerCollection<T> that behaves like a set (e.g. no duplicates allowed). Here is a simplistic version that use a critical section (C# lock) to make it thread-safe. For high concurrency scenarios you may be able to improve performance by using a class like SpinWait the same way as ConcurrentQueue<T> does.

public class ProducerConsumerSet<T> : IProducerConsumerCollection<T> {

  readonly object gate = new object();

  readonly Queue<T> queue = new Queue<T>();

  readonly HashSet<T> hashSet = new HashSet<T>();

  public void CopyTo(T[] array, int index) {
    if (array == null)
      throw new ArgumentNullException("array");
    if (index < 0)
      throw new ArgumentOutOfRangeException("index");
    lock (gate)
      queue.CopyTo(array, index);
  }

  public bool TryAdd(T item) {
    lock (gate) {
      if (hashSet.Contains(item))
        return false;
      queue.Enqueue(item);
      hashSet.Add(item);
      return true;
    }
  }

  public bool TryTake(out T item) {
    lock (gate) {
      if (queue.Count == 0) {
        item = default(T);
        return false;
      }
      item = queue.Dequeue();
      hashSet.Remove(item);
      return true;
    }
  }

  public T[] ToArray() {
    lock (gate)
      return queue.ToArray();
  }

  public void CopyTo(Array array, int index) {
    if (array == null)
      throw new ArgumentNullException("array");
    lock (gate)
      ((ICollection) queue).CopyTo(array, index);
  }

  public int Count {
    get { return queue.Count; }
  }

  public object SyncRoot {
    get { return gate; }
  }

  public bool IsSynchronized {
    get { return true; }
  }

  public IEnumerator<T> GetEnumerator() {
    List<T> list = null;
    lock (gate)
      list = queue.ToList();
    return list.GetEnumerator();
  }

  IEnumerator IEnumerable.GetEnumerator() {
    return GetEnumerator();
  }

}

If required you can elaborate on this class to customize equality by supplying an optional IEqualityComparer<T> that then is used to initialize the HashSet<T>.

The IProducerConsumerCollection<T>.Add methods returns false when there is an attempt to insert a duplicate item. This results in an InvalidOperationException thrown by the BlockingCollection<T>.Add method so you will probably have to wrap the code to add an item into something like this:

bool AddItem<T>(BlockingCollection<T> blockingCollection, T item) {
  try {
    blockingCollection.Add(item);
    return true;
  }
  catch (InvalidOperationException) {
    return false;
  }
}

Note that if you add items to a collection that has been completed you will also get an InvalidOperationException and you will have to examine the exception message to determine the underlying reason for the exception.