How to "adopt" a child process in python

Question

How can I get the pid of a child processes that I didn't create an object for? i.e.

myProc = Popen(["sleep","30"])

vs

Popen(["sleep","30"])

I've noticed they become zombie processes if I don't poll() or wait() on them after sending a termination signal. At a point in my script I would like to find all child processes that my script is a parent of and send them a signal or poll them. Is this possible in python? Is it possible at all?

Answer 1

You could use psutil to find the children of your parent Python process. For example:

import psutil
import os
import subprocess

subprocess.Popen(['sleep', '30'])

parent_pid = os.getpid()
parent = psutil.Process(parent_pid)

for child in parent.children():
    print(child)    # do something here

Prints:

psutil.Process(pid=16822, name='sleep')

From there you could poll them, kill them etc.

Answer 2

Normally, you don't need to do anything -- the current subprocess implementation maintains a global list of active unreferenced Popen instances and when a new Popen object is created, this list is enumerate and .poll() is called for each process.

Thus if you don't have a reference to the subprocess; it is waited for you automatically (and if you do have a reference then call .wait() yourself).

If child processes are created by other means then you could call os.waitpid() to collect exit statuses of dead subprocesses on Unix:

while True:
    try:
        pid, status = os.waitpid(-1, os.WNOHANG) 
    except ChildProcessError:
        #  No more child processes exist
        break
    else:
        assert pid, "child process is still alive"

On POSIX.1-2001 systems, you could call signal(SIGCHLD, SIG_IGN) to reap children automatically instead.

If you want to kill all children (send a signal) when a parent dies; you could use prctl PR_SET_PDEATHSIG on Linux. It works if the parent dies for any reason i.e., it works even if the parent is killed by SIGKILL.

psutil from @ali_m' answer is a portable solution:

#!/usr/bin/env python
import gc
import subprocess
import time

import psutil


for _ in range(10):
    subprocess.Popen(['sleep', '1'])  # no reference
time.sleep(2)  # wait until they are dead
gc.collect()  # run garbage collection, to populate _active list
subprocess.Popen(['sleep', '1'])  # trigger _cleanup()

for i in range(2):
    for child in psutil.Process().children():  # immediate children
        print(child, child.status())
    if i == 0:
        time.sleep(2)

Output

psutil.Process(pid=31253, name='sleep') sleeping
psutil.Process(pid=31253, name='sleep') zombie

Note:

1. psutil shows only one zombie process, the rest are reaped in the last Popen() call
2. psutil provides a protection against pid reuse but it is not 100% reliable in all cases -- check whether it is enough in your case (otherwise, use one the methods above that do not rely on child's pid).