Read shell output from python line-by-line

Question

I need to read a shell command output from python script and fill a list with each line:

ISSUE: If I save the command output as a variable, the loop cycle read it character-by-character

#!/usr/bin/python
import subprocess

CMD = "ls -la"
output = subprocess.check_output(CMD, shell=True)
list = []
for line in output:
    list.append(line)
print list

Bad output: ['t', 'o', 't', 'a', 'l', ' ', '5',...]

WORKAROUND: As a work around, I directed the comand output to a file, and from there read line-by-line:

#!/usr/bin/python
import subprocess

CMD = "ls -la > tmp"
subprocess.call(CMD, shell=True)
list = []
f = open("tmp", "r")
for line in f:
    list.append(line)
print list

Good output ['total 52\n', 'drwxrwxr-x 3 ajn ajn 4096 mars 11 17:52 .\n',...]

QUESTION: How can I use the 1st approach (no file manipulation ,all inside the program) with the result of the second approach?

Answer 1

I'm not really a python guy, but this should work for you and seems a bit clearer:

import subprocess
cmd = ["ls", "-la"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
for line in proc.stdout.readlines():
    print line

Answer 2

Don't use proc.stdout.readlines(). This reads all OUTPUT before it returns. You need to use proc.stdout.

Here is yy.py:

import subprocess
cmd = ["bash", "./yy.sh"]

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
for line in proc.stdout:
    print( line )

Here is yy.sh:

for a in 1 2 3 4 5 6 7 8 9 0; do
    echo "hi $a !!!"
    sleep 1
done

When you run yy.py you get output line by line.

If you use proc.stdout.readlines(), you wait 10 seconds and get all output.