stdin should not wait for "CTRL+D"

Question

I got a simple python script which should read from stdin. So if I redirect a stdout of a program to the stdin to my python script.

But the stuff that's logged by my program to the python script will only "reach" the python script when the program which is logging the stuff gets killed.

But actually I want to handle each line which is logged by my program as soon as it is available and not when my program which should actually run 24/7 quits.

So how can I make this happen? How can I make the stdin not wait for CTRL+D or EOF until they handle data?

Example

# accept_stdin.py
import sys
import datetime

for line in sys.stdin:
    print datetime.datetime.now().second, line

# print_data.py
import time

print "1 foo"
time.sleep(3)
print "2 bar"

# bash
python print_data.py | python accept_stdin.py

Answer 1

Like all file objects, the sys.stdin iterator reads input in chunks; even if a line of input is ready, the iterator will try to read up to the chunk size or EOF before outputting anything. You can work around this by using the readline method, which doesn't have this behavior:

while True:
    line = sys.stdin.readline()
    if not line:
        # End of input
        break
    do_whatever_with(line)

You can combine this with the 2-argument form of iter to use a for loop:

for line in iter(sys.stdin.readline, ''):
    do_whatever_with(line)

I recommend leaving a comment in your code explaining why you're not using the regular iterator.