Saving pprofile output to a file

Question

I am using pprofile to do line profiling on a python script. However, since pprofile also profiles all the modules called by my script, the output is extremely long and gets cut off in the command line. I am only interested in the output of profiling my script, which is at the beginning of the output and thus gets cut off. Is there a way to either:

1. Tell pprofile to only output the results of profiling my script, and none of the other modules, or
2. Save the pprofile output to a file so that I can see the beginning of it?

Answer 1

1. Tell pprofile to only output the results of profiling my script, and none of the other modules, or

There are the --exclude-syspath, --exclude some_regex and --include some_regex options. For simple setups they should be rather straightforward to use.

Some background:

• How to match files which do not exist on the filesystem ? Base your regex on the name the code you are interested in appears as in pprofile output. The matched value may look like a filename, but it is actually co_filename, which can be an arbitrary value.
• Why regexes and not globbing ? As the value may not be a path, I considered that globbing may not be applicable, whereas regexes could fit the task better.

2. Save the pprofile output to a file so that I can see the beginning of it?

There is the -o/--output option to direct the output to a file instead of stdout. Depending on your terminal/shell, you could also use stream redirection operations, typically >: pprofile some_script.py > pprofile_output.txt.

And going further, on larger programs I recommend using --format callgrind and qcachegrind/kcachegrind (depending on your OS) to visualize the profiling result in a much more convenient way than plain concatenated source code. Combined with --zipfile it has the added benefit of extracting the profiled source code, should you be profiling code on a headless machine and analyzing the output on a different one.