How to capture interlaced stdin and stdout from a Python subprocess

Question

I am trying to write a Python script that automatically grades a Python script submitted by a student, where the student's script uses the input() function to get some information from the user.

Suppose the student's script is something simple like this:

name = input('Enter your name: ')
print(f'Hello {name}!')

The portion of the test script that runs the student script is something like this:

import subprocess

run_cmd = 'python student_script.py'
test_input = 'Bob'

p = subprocess.run(run_cmd.split(), input=test_input, capture_output=True, text=True)

After running that portion of the test script, output from the student's script is captured and can be accessed via p.stdout which is a string having this value:

'Enter your name: Hello Bob!\n'

No surprise there, since this is everything output by the student script, but notice that the 'Bob' test input is not included.

In the test report, I want to show the script input and output in the same way that it would appear if the script had been run from a command line, which would look like this:

Enter your name: Bob
Hello Bob!

Given that the scripts are written by students, the prompt message output by the student script could be anything (e.g., What is your name?, Who are you?, Type in name:, etc.) and the student script might also print something other than 'Hello Bob!', so I don't think there is any way to reliably figure out where to correctly insert the 'Bob' test input (and a trailing new line) into p.stdout.

Is there a way to get subprocess.run() to capture interlaced stdin and stdout?

Or is there another way to run a Python script from a Python script that captures interlaced stdin and stdout?

Ideally, for this example, I would be able to get a string having this value:

'Enter your name: Bob\nHello Bob!\n'

I've search SO and read through the subprocess documentation, but thus far I've come up short on finding a solution.

Answer 1

Here's the solution I came up with. I expect there is a more elegant way to do it, but it works on the Ubuntu Linux computer that the automated test scripts run on. I have not tried it on Windows, but I believe it will not work since os.set_blocking() is only supported on Unix per the os module documentation.

import subprocess
import os
import time

run_cmd = 'python student_script.py'
test_input = 'Bob'

# Start the student script running
p = subprocess.Popen(run_cmd.split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE, text = True)

# Give the script some time to run
time.sleep(2)

# String to hold interleaved stdin and stdout text
stdio_text = ''

# Capture everything from stdout
os.set_blocking(p.stdout.fileno(), False)  # Prevents readline() blocking
stdout_text = p.stdout.readline()
while stdout_text != '':
    stdio_text += stdout_text
    stdout_text = p.stdout.readline()

# Append test input to interleaved stdin and stdout text
stdio_text += (test_input + '\n')

try:
    # Send test input to stdin and wait for student script to terminate
    stdio_text += p.communicate(input=test_input, timeout=5)[0]
except subprocess.TimeoutExpired:
    # Something is wrong with student script
    pass

p.terminate()

The key to this solution working is os.set_blocking(), which I found out about here. Without it readline() blocks indefinitely.

I don't love the time.sleep(2) since it assumes it will take 2 seconds or less for the student script to reach the point where it calls input(), but there does not seem to be any way to determine when a process is looking for input from stdin. The sleep time could be increased for longer scripts.

If you've got any ideas for improvements, please share.