Visualising a Python algorithm profile

Writing code is a delicate task, as one misplaced comma leads to compiler displays weird error messages. A practice that I usually employ while coding is to write the whole algorithm as a combination of smaller components (snippets).

A positive point for dividing it into components is the ability to debug individual parts of the code, as it is possible to run each part almost independently. A negative point is the creation of numerous components. When the number of snippets becomes too large, it is hard to keep track of their relationships, compromising the advantage aforementioned.

Another procedure to test the quality of code is to analyse its profile. Profiling is the process of measuring,

for example, the space (memory) or time complexity of a program, the usage of particular instructions, or the frequency and duration of function calls. [1]

In Python, the profiling program

cProfile 

provides some statistics including the number of times a given instruction has been called and the time taken to execute it.

Assuming that you want to profile the file script.py and save the statistics to the file script.prof, the profiler should be executed as follows

python -m cProfile -o script.prof script.py

The created file, script.prof is not human readable. You need an interpreter to be able to see the statistics. For that case, you need to write a second. The following lines of code will do the job:

import pstats
f = open("script.txt", "w")
stats = pstats.Stats('script.prof, stream=f)
stats.print_stats()

Note that the statistics are now saved in the file script.txt. This file will have a content that is similar to the following

Fri May 19 15:36:07 2017 script.txt

9497977 function calls (9050659 primitive calls) in 15.824 seconds

Random listing order was used

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/matplotlib/patches.py:2058(LArrow)
2 0.000 0.000 0.000 0.000 {method 'find_module' of 'zipimport.zipimporter' objects}
1 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/scipy/optimize/_basinhopping.py:36(BasinHoppingRunner)
1409 0.002 0.000 0.020 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py:2530(_get_metadata)
3 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.py:231(set_ylabel)
1 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/decimal.py:160(DecimalException)
1 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/six.py:451(Module_six_moves_urllib_robotparser)
1 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/matplotlib/collections.py:1547(EllipseCollection)
233 0.000 0.000 0.000 0.000 /home/humberto/anaconda2/lib/python2.7/site-packages/matplotlib/backend_bases.py:1120(get_sketch_params)

Plase, note that I have added only some of the lines. A better to way to inspect the code profile is to use a visualisation tool such as snakeviz. This tool provides charts with the time consumed by each element of the script.

20170520

I have executed for a script in which I plot some graphs. The pyplot module takes about 31% of the total execution time.

To use snakeviz, just run

snakeviz script.prof

and the program will generate a web page to visualise the graph.

References

[1] Unkown, Profiling (computer programming), https://en.wikipedia.org/wiki/Profiling_(computer_programming), last access: May 2017,

 

One thought on “Visualising a Python algorithm profile

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s