Python queue.get(block=true) with timeout does not return when item is added

Question

I have 2 threads in my python application. Thread A (well, basically the 'main' thread) is added items to the queue. Thread B is getting it from the queue.

Code A:

def addTrade(self, date, volume, price, exchange):
    '''
    Adds a single trade to the database
    '''
    print "> ADD"
    try:
        self._incomingDataQueue._put(TradeData(exchange=exchange, date=date, volume=volume, price=price))
        # self._dataAvailableEvent.set()
        # self._dataAvailableEvent.clear()
        print "< ADD"
    except Exception as ex:
        print "ex: %s" % ex

Thread B has this:

print "> GET"
t = int(time.time())
tradeData = self._incomingDataQueue.get(block=True, timeout=20)
print "< GET %d " % (int(time.time()) - t)

So ...

What happens is this: Thread B is started and waits for an item in the Queue (timeout = 20 seconds). Almost instantaneously after B started, an item is added to the Queue. 15 seconds after that another item.

However: the get on the Queue only retunrs after 20 seconds. I would expect it to return 'nearly instant' when new data is available.

Output:

> GET
> ADD
< ADD
> ADD
< ADD
< GET 20

So, is this normal behaviour for a Queue? Or should I use another mechanism ?

Thanks in advance !

Answer 1

The problem is that you are calling Queue._put instead of Queue.put. I see no reason why you'd want to do that.

The leading underscore indicates the _put method is not part of the public interface of the class Queue, so you should not be calling it directly.