How to use SyncManager.Lock or Event correctly?

Question

I'm having trouble using SyncManager.Lock correctly. I read the official doc, but it offers no working example. I also have no idea how to use SyncManager.Event correctly.

Below is the minimal code to illustrate my problem. client1 and client2 both need to update a shared object Struct. However, I want client1 to acquire the lock first, update Struct, and then pass control to client2. If you run the code below as-is, the print statements are all mixed up.

import multiprocessing as mp
from multiprocessing.managers import SyncManager
import time

class Struct:
    def __init__(self):
        self.a = []

    def update(self, x, y):
        self.a.append(x ** 2)

    def get(self):
        return self.a

class Server(SyncManager):
    pass

global_S = Struct()
Server.register('Struct', lambda: global_S)

def server_run():
    print('Server starting ...')
    manager = Server(('localhost', 8080), authkey=b'none')
    manager.get_server().serve_forever()


def client_run(name, x, y, wait):
    server_proc = Server(('localhost', 8080), authkey=b'none')
    server_proc.connect()
    S = server_proc.Struct()
    with server_proc.Lock():
        for i in range(5):
            S.update(x+i, y+i)
            print(name, S.get())
            time.sleep(wait)


server = mp.Process(target=server_run)
server.daemon = True

client1 = mp.Process(target=client_run, args=('c1', 3,7, 1))
client2 = mp.Process(target=client_run, args=('c2', 100,120, .6))

server.start()
time.sleep(0.3) # wait for server to spawn up
client1.start()
time.sleep(0.3)
client2.start()

client1.join()
client2.join()

Sample output:

Server starting ...
c1 [9]
c2 [9, 10000]
c2 [9, 10000, 10201]
c1 [9, 10000, 10201, 16]
c2 [9, 10000, 10201, 16, 10404]
c1 [9, 10000, 10201, 16, 10404, 25]
c2 [9, 10000, 10201, 16, 10404, 25, 10609]
c2 [9, 10000, 10201, 16, 10404, 25, 10609, 10816]
c1 [9, 10000, 10201, 16, 10404, 25, 10609, 10816, 36]
c1 [9, 10000, 10201, 16, 10404, 25, 10609, 10816, 36, 49]

Answer 1

I figured out a workaround. Don't use the builtin SyncManager.Lock() for the following reasons:

1. It's creating a new Lock object every time instead of sharing.
2. It wraps around threading.Lock(), NOT multiprocess.Lock(). Looks like it doesn't work with multiprocessing!

Solution is to register your own lock manager:

from multiprocessing.managers import BaseManager, AcquirerProxy
global_lock = mp.Lock()

def get_lock():
    print('getting global_lock')
    return global_lock

Server.register('Lock', get_lock, AcquirerProxy)