calling awk from python

Question

i wanted to call an awk commandline script from python:

os.system('''awk 'BEGIN{FS="\t";OFS="\n"} {a[$1]=a[$1] OFS $2 FS $3 FS $4} END{for (i in a) {print i a[i]}}' 2_lcsorted.txt > 2_locus_2.txt''')

it gives the following error:

awk: cmd. line:1: BEGIN{FS="    ";OFS="
awk: cmd. line:1:                     ^ unterminated string
awk: cmd. line:1: BEGIN{FS="    ";OFS="
awk: cmd. line:1:                     ^ syntax error
256

when I use subprocess using subprocess.call, another kind of error pops:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib64/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

it runs fine in the shell and all i want to do is to combine all steps in a single python script and for some obvious reasons awk is better for certain processing steps. Can someone please explain me the cause of these errors ?

Answer 1

You do not want Python to convert \n to a newline character (or \t to a tab) before feeding the string to system. Use r"""....""" as jwpat7 suggested. Another possibility is to write something like ... OFS="\\n" ... in the string.