WinError6 The Handle is Invalid Python 3+ multiprocessing

Question

I am running a Python 3.7 Flask application which uses flask_socketio to setup a socketio server for browser clients, another python process to connect to a separate remote socketio server & exchange messages, and another python process to read input from a PIR sensor.

Both python processes communicate over multiprocessing.Queue - but, the socketio process always gets either [WinError6] - Invalid Handle or [WinError5] - Permission Denied. I have absolutely no idea what I'm doing wrong.

Here's the top-level (server) code; it does not appear to have issues:

from shotsocket import init as shotsocket_init
from shotsocket import util as matchmaking_util
import multiprocessing, os, config, uuid

match_queue = multiprocessing.Queue()
shot_queue = multiprocessing.Queue()

app = Flask(__name__, static_url_path='', static_folder='templates')
socketio = SocketIO(app)

_rooms = [] # I don't plan to keep this in memory, just doing it for debug / dev 

...

The above works fine and dandy. The 2nd to last line in the following block is the issue.

# THIS IS THE FUNC WHERE WE ARE TRYING TO USE 
# THE BROKEN QUEUE

@socketio.on('connect')
def listen():
    room_key = str(uuid.uuid4())
    join_room(room_key)
    _rooms.append((room_key, request.sid))
    possible_match = matchmaking_util.match_pending_clients(_rooms)
    if possible_match:
        shot_queue.put_nowait(possible_match)
        print('put it in there')

Here's how I start these processes:

if __name__ == '__main__':
    debug = os.environ.get('MOONSHOT_DEBUG', False)
    try:
        proc = multiprocessing.Process(target=start, args=(debug,match_queue))
        proc.start()
        shot_proc = multiprocessing.Process(target=shotsocket_init, args=(shot_queue,))
        shot_proc.start()
        socketio.run(app, host='0.0.0.0')
    except KeyboardInterrupt:
        socketio.stop()
        proc.join()
        shot_proc.join()

And here's the entirety of shotsocket (the code that cannot read the queue)

import socketio, multiprocessing # mp for the type

sio = socketio.Client(engineio_logger=True)
sio.connect('redacted woot', transports=['websocket'])


@sio.on('connect')
def connect():
    print("connected to shot server")

def init(queue: multiprocessing.Queue):
    while True:
        try:
            # WE NEVER GET PAST THIS LINE
            print(queue.get())
        except Exception as e:
            continue
        if not queue.empty():
            print('queue empty')
            shot = queue.get()
            print(shot)         
            match_id, opponents = shot
            sio.emit('start', {'id': match_id, 'opponents': [opponents[0], opponents[1]]})

I'm pulling my hair out. What the heck am I doing wrong?

Answer 1

Solution

I have no idea why this fixes the problem, but switching from multiprocessing.Queue to queue.Queue and multiprocessing.Process to threading.Thread did it.