C program and subprocess

Question

I wrote this simple C program to explain a more hard problem with the same characteristics.

#include <stdio.h>

int main(int argc, char *argv[])
{
    int n;
    while (1){
        scanf("%d", &n);
        printf("%d\n", n);
    }
    return 0;
}

and it works as expected.

I also wrote a subprocess script to interact with this program:

from subprocess import Popen, PIPE, STDOUT

process = Popen("./a.out", stdin=PIPE, stdout=PIPE, stderr=STDOUT)

# sending a byte
process.stdin.write(b'3')
process.stdin.flush()

# reading the echo of the number
print(process.stdout.readline())

process.stdin.close()

The problem is that, if I run my python script, the execution is freezed on the readline(). In fact, if I interrupt the script I get:

/tmp » python script.py
^CTraceback (most recent call last):
  File "/tmp/script.py", line 10, in <module>
    print(process.stdout.readline())
          ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

If I change my python script in:

from subprocess import Popen, PIPE, STDOUT

process = Popen("./a.out", stdin=PIPE, stdout=PIPE, stderr=STDOUT)

with process.stdin as pipe:
    pipe.write(b"3")
    pipe.flush()

# reading the echo of the number
print(process.stdout.readline())

# sending another num:
pipe.write(b"4")
pipe.flush()

process.stdin.close()

I got this output:

» python script.py
b'3\n'
Traceback (most recent call last):
  File "/tmp/script.py", line 13, in <module>
    pipe.write(b"4")
ValueError: write to closed file

That means that the first input is sent correctly, and also the read was done.

I really can't find something that explain this behaviour; can someone help me understanding? Thanks in advance

[EDIT]: since there are many points to clearify, I added this edit. I'm training on exploitation of buffer overflow vuln using the rop technique and I'm writing a python script to achieve that. To exploit this vuln, because of ASLR, I need to discover the libc address and make the program restart without terminating. Since the script will be executed on a target machine, I dont know which libraries will be avaiable, then I'm going to use subprocess because it's built-in in python. Without going into details, the attack send a sequence of bytes on the first scanf aim to leak libc base address and restart the program; then a second payload is sent to obtain a shell with which I will communicate in interactive mode.

That's why:

1. I can only use built-in libraries
2. I have to send bytes and cannot append ending \n: my payload would not be aligned or may leeds to fails
3. I need to keep open the stdin open
4. I cannot change the C-code

Answer 1

I really can't find something that explain this behaviour; can someone help me understanding?

There are at least two issues here:

1. the C program's standard output is to a non-interactive device (a pipe) and therefore the output is fully buffered. The output will be accumulated in memory, only being sent to the output device when (i) the buffer fills, or (ii) it is manually flushed, or (iii) the program terminates normally.

2. the C program runs indefinitely, with no way to make it terminate normally. Not even end-of-file on its standard input will make it stop.

On the Python side, then, the script is waiting for the subprocess's output to be flushed, whereas the subprocess needs to read and echo more data (probably a lot of it) before that will happen. This is a deadlock.

Your best approach depends on how well the example C program models the behavior of the one in which you're really interested.

• If it's faithful in every relevant detail, then the only thing you can reasonably do is keep writing to the subprocess's stdin until something pops out on its stdout. You probably want to do the data pushing in a separate thread, to avoid reproducing exactly the same kind of deadlock you already saw.

• If it recognizes some kind of clean termination signal, such as a "quit" command or EOF on its standard input, then have the Python side send that signal before trying to read the output. Indeed, you already close stdin, so perhaps you don't actually need to do anything to accommodate the real program.

Note well that the first alternative leaves the subprocess running. You will want to ensure that it terminates. By terminate()ing or kill()ing it, if that's what it takes.

Answer 2

Change these:

• Send a separator between the numbers read by the C program. scanf(3) accepts any non-digit byte as separator. For easiest buffering, send a newline (e.g. .write(b'42\n')) from Python. Without a separator, scanf(3) will wait for more digits indefinitely.

• After each write (both in C and Python), flush the output.

This works for me:

#include <stdio.h>

int main(int argc, char *argv[])
{
    int n;
    while (1){
        scanf("%d", &n);
        printf("%d\n", n);
        fflush(stdout);  /* I've added this line only. */
    }
    return 0;
}

import subprocess

p = subprocess.Popen(
    ('./a.out',), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
try:
  print('A'); p.stdin.write(b'42 '); p.stdin.flush()
  print('B'); print(repr(p.stdout.readline()));
  print('C'); p.stdin.write(b'43\n'); p.stdin.flush()
  print('D'); print(repr(p.stdout.readline()));
finally:
  print('E'); print(p.kill())

The reason why your original C program works when run interactively within the terminal window is that in C the output is automatically flushed when a newline (\n) is written to the terminal. Thus printf("%d\n", n); does an implicit fflush(stdout); in the end.

The reason why your original C program doesn't work when run from Python with subprocess is that it writes its output to a pipe (rather than to a terminal), and there is no autoflush to a pipe. What happens is that the Python program is waiting for bytes, and the C program doesn't write those bytes to the pipe, but it is waiting for more bytes (in the next scanf), so both programs are waiting for the other indefinitely. (However, there would be a partial autoflush after a few KiB (typically 8192 bytes) of output. But a single decimal number is too short to trigger that.)

If it's not possible to change the C program, then you should use a terminal device instead of a pipe for communication between the C and the Python program. The pty Python module can create the terminal device, this works for me with your original C program:

import os, pty, subprocess

master_fd, slave_fd = pty.openpty()
p = subprocess.Popen(
    ('./a.out',), stdin=slave_fd, stdout=slave_fd,
    preexec_fn=lambda: os.close(master_fd))
try:
  os.close(slave_fd)
  master = os.fdopen(master_fd, 'rb+', buffering=0)
  print('A'); master.write(b'42\n'); master.flush()
  print('B'); print(repr(master.readline()));
  print('C'); master.write(b'43\n'); master.flush()
  print('D'); print(repr(master.readline()));
finally:
  print('E'); print(p.kill())

If you don't want to send newlines from Python, here is a solution without them, it works for me:

import os, pty, subprocess, termios

master_fd, slave_fd = pty.openpty()
ts = termios.tcgetattr(master_fd)
ts[3] &= ~(termios.ICANON | termios.ECHO)
termios.tcsetattr(master_fd, termios.TCSANOW, ts)
p = subprocess.Popen(
    ('./a.out',), stdin=slave_fd, stdout=slave_fd,
    preexec_fn=lambda: os.close(master_fd))
try:
  os.close(slave_fd)
  master = os.fdopen(master_fd, 'rb+', buffering=0)
  print('A'); master.write(b'42 '); master.flush()
  print('B'); print(repr(master.readline()));
  print('C'); master.write(b'43\t'); master.flush()
  print('D'); print(repr(master.readline()));
finally:
  print('E'); print(p.kill())