'yes' reporting error with subprocess communicate()

Question

I am using the following function to run a command in Python:

def run_proc(cmd):
    child = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdout, stderr = child.communicate()
    returncode = child.returncode
    return stdout, stderr, returncode

It has always been working fine, however now I'm trying to use the yes program to pipe output to stdin. The command I'm trying to run is the following:

yes '' | apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

but I believe it could be substituted with a general example, like:

yes | head -3 | cat

My problem is that if I try to run any command which has yes | in it, the above subprocess.Popen will contain the error messages:

yes: standard output: Broken pipe
yes: write error

For me it seems that the piping still works, as can be seen from yes | head -3 | cat's answer: y y y.

I have the following questions:

1. Is the yes piping still functional, even though yes reports error?
2. How can I fix it?

Answer 1

The issue is that subprocess module before Python 3.2+ doesn't restore SIGPIPE signal handler to default action. That is why you get EPIPE write error instead.

In Python 3.2+

>>> from subprocess import check_output
>>> check_output("yes | head -3", shell=True)
b'y\ny\ny\n'

yes is killed by SIGPIPE when head exits.

In Python 2:

>>> from subprocess import check_output
>>> check_output("yes | head -3", shell=True)
yes: standard output: Broken pipe
yes: write error
'y\ny\ny\n'

yes got EPIPE write error. It is safe to ignore the error. It communicates the same information as SIGPIPE.

To workaround the problem, you could emulate restore_signals in Python 2 using preexec_fn parameter :

>>> from subprocess import check_output
>>> import signal
>>> def restore_signals(): # from http://hg.python.org/cpython/rev/768722b2ae0a/
...     signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
...     for sig in signals:
...         if hasattr(signal, sig):
...            signal.signal(getattr(signal, sig), signal.SIG_DFL)
... 
>>> check_output("yes | head -3", shell=True, preexec_fn=restore_signals)
'y\ny\ny\n'