Why are system calls so much slower in Python compared to C++?

Question

I have 2 code snippets that are identical in the sense that they complete the same task. One code was written in python, the other in C++. All they do is call an executable that generates an ASCII file. In C++, I use the system() command to call the executable. In Python, I have used many things including os.system subprocess.call subprocess.popen.

I realize that C++ is a compiled language while Python is interpreted, and therefore, Python calls have more overhead. But the C++ code does the job nearly 100 times faster than the Python code. The C++ took about 0.004 seconds but the Python took around 0.35 seconds.

Even a simple pwd command takes more than 10 times longer with Python than it does with C++. If the overhead is what is slowing the Python code down, is there a faster option in Python than what I have already tried?

Here is the Python code:

from os import system
from time import time

t0 = time();
system("pwd");
print "duration: ",time()-t0;

Here is the same thing in C++:

#include <iostream>
#include <sys/time.h>
double diff(timespec start, timespec end) { return (end.tv_nsec-start.tv_nsec)/1e9; }

int main()
{
    timespec t0, t1;
    clock_gettime(CLOCK_PROCESS_CPUTIME_ID, & t0);
    system("pwd");
    clock_gettime(CLOCK_PROCESS_CPUTIME_ID, & t1);
    
    std::cout << "duration: " << diff(t0,t1) << "\n";
    
    return 0;
}

I used GCC to compile the C++ code. You have to use the -lrt option to get the code to compile correctly.

You can run the code yourself. My timing methods could be wrong. But if they are correct, then the Python script takes more than 10 times as long to execute the pwd command compared to the C++ executable.

Answer 1

C 'exec' call executes the program directly.

While the Python 'system' call first executes bash, that executes the program in question.

Answer 2

I cooked up a little script and execution time was much faster than what you are seeing.

td@timsworld2:~/tmp/so$ cat nothing.py
#!/usr/bin/env python
import subprocess
import sys

cmd = ['python', '-V'] if 'py' in sys.argv else ['pwd']
if 'shell' in sys.argv:
    subprocess.call(' '.join(cmd), shell=True)
else:
    subprocess.call(cmd)


td@timsworld2:~/tmp/so$ time ./nothing.py
/home/td/tmp/so

real    0m0.024s
user    0m0.012s
sys     0m0.008s
td@timsworld2:~/tmp/so$ time python nothing.py
/home/td/tmp/so

real    0m0.020s
user    0m0.012s
sys     0m0.004s
td@timsworld2:~/tmp/so$ time ./nothing.py py
Python 2.7.3

real    0m0.022s
user    0m0.016s
sys     0m0.000s
td@timsworld2:~/tmp/so$ time ./nothing.py sh
/home/td/tmp/so

real    0m0.020s
user    0m0.012s
sys     0m0.004s
td@timsworld2:~/tmp/so$