How to avoid zombie processes in python?

Question

In Python3, I have essentially the following code:

server.py:

import os
import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("127.0.0.1", 10000))
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.listen(5)

while True:
    print("waiting")
    connection, client_address = sock.accept()
    print("received")
    child_pid = os.fork()
    if child_pid == 0:
        print("connection received")
        received = connection.recv(1024)
        connection.sendall("OK".encode('utf-8'))
        os._exit(0)

client.py:

import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(("127.0.0.1", 10000))    
sock.close()

When I start the server and then the client, each time the client finishes a zombie process remains.

How to change the code so that no zombie process remains?

Answer 1

The usual technique is to track all the child pids so that they can be killed when the main process exits or whenever you want the children to be cleaned-up.

You can periodically poll and reap processes as needed or wait until you're about to exit.

For an example of how to do this, look at the collect_children() code in the ForkingMixin for the SocketServer module.

The os module has a number of tools for managing the subprocesses like os.wait() and os.kill.

I don't know whether it fits your problem or not, but a multiprocessing.Pool() may be of some help. It automatically manages a pool of subprocesses and reuses them for future tasks. It is mainly helpful when there is only limited data exchange between the processes and whether the work is relatively homogenous (all the child processes are doing the same kind of work).