python sys.stdin.read() from tail -f

Question

How come sys.stdin.read() doesn't read the piped input from tail -f?

#!/usr/bin/env python
import sys
from geoip import geolite2
def iplookup(srcip):
        for ip in srcip.split("\n"):
                try:
                        print(geolite2.lookup(ip))
                except:
                        pass
source = sys.stdin.read()
iplookup(source)

tail -f /var/log/bleh.log | grep -oE '((1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])' | python mygeoip.py

Answer 1

You can use fileinput :

import sys
from geoip import geolite2
import fileinput

def iplookup(srcip):
        for ip in srcip.split("\n"):
                try:
                        print(geolite2.lookup(ip))
                except:
                        pass

for line in fileinput.input():
    iplookup(line)

On the plus side, your script automagically accepts filename as parameters as well.

Answer 2

None of the other answers (even fileinput) fully addresses the issue of buffering, and so will not work for small outputs of tail -f.

From the python man page:

Note that there is internal buffering in xreadlines(), readlines() and file-object iterators ("for line in sys.stdin") which is not influenced by this option. To work around this, you will want to use "sys.stdin.readline()" inside a "while 1:" loop.

In other words what you want is:

while True:
    line = sys.stdin.readline()
    iplookup(line)