Python 3.6: "AttributeError: 'SimpleQueue' object has no attribute '_poll'"

Question

When I use a standard Queue to send samples to a process, everything works fine. However, since my needs are simple, I tried to use a SimpleQueue and for some reason the 'empty' method doesn't work. Here's the details:

Error comes from the consumer process (when sample_queue is Queue, everything works, when sample_queue is SimpleQueue, things break):

      def frame_update(i):
        while not self.sample_queue.empty():
            sample = self.sample_queue.get()
            for line in lines:

While executing sample_queue.empty() -- SimpleQueue.empty(), from Python 3.6 on windows (queues.py) we get:

    def empty(self):
      return not self._poll()

Where self._poll() has been set in init by:

  def __init__(self, *, ctx):
    self._reader, self._writer = connection.Pipe(duplex=False)
    self._rlock = ctx.Lock()
    self._poll = self._reader.poll
    if sys.platform == 'win32':
        self._wlock = None
    else:
        self._wlock = ctx.Lock()

So I follow the self._reader which is set from connection.Pipe (connection.py): ...

c1 = PipeConnection(h1, writable=duplex)
c2 = PipeConnection(h2, readable=duplex)

Ok, great. The _reader is going to be a PipeConnection and pipe connection has this method:

      def _poll(self, timeout):
        if (self._got_empty_message or
                    _winapi.PeekNamedPipe(self._handle)[0] != 0):
            return True
        return bool(wait([self], timeout))

Alright -- So a couple of questions:

1) Shouldn't the init of SimpleQueue be assigning self.poll to self._reader._poll instead of self._reader.poll? Or am I missing something in the inheritance hierarchy?

2) The PipeConnection _poll routine takes a timeout parameter, so #1 shouldn't work...

*) -- Is there some other binding of PipeConnection _poll that I'm missing?

Am I missing something? I am using Python3.6, Windows, debugging in PyCharm and I follow all the paths and they're in the standard multiprocessing paths. I'd appreciate any help or advice. Thanks!

EDIT: After further review, I can see that PipeConnection is a subclass of _ConnectionBase which does indeed have a 'poll' method and it is bound with a default timeout parameter.

So the question is: When SimpleQueue is initializing and sets

self._poll = self._reader.poll

Why doesn't it go up the class hierarchy to grab that from _ConnectionBase?

Answer 1

After looking at why the Queue type works and why the SimpleQueue doesn't, I found that Queue sets the _poll method 'after_fork' as well as before. SimpleQueue doesn't. By changing the setstate method to add self._poll = self._reader.poll as follows (queues.py, line 338), SimpleQueue works

def __setstate__(self, state):
    (self._reader, self._writer, self._rlock, self._wlock) = state
    self._poll = self._reader.poll

Seems like a bug to me unless I'm really misunderstanding something. I'll submit a bug report and reference this post. Hope this helps someone!

http://bugs.python.org/issue30301