Tools for developing with the Python programming language
This page summarizes auxiliary Python tools that we commonly use to develop the project. Note that this page is meant to be a quick index. Consult the documentation on each specific tool for details.
The Spyder editor is good for getting started with scientific Python coding, because of various inspection and interactive features.
vscodeis a more full featured editor.
In the long run, the most efficient approach is to learn a terminal based editor such as
vim. Note that
vimediting modes can be added as extensions to graphical editors such as
Python code can be evaluated interactively, which can speed up the development.
IPython shell: It is notably nicer to use than the standard interactive interpreter.
Jupyter notebook: Interactive development environment running on the browser. Useful for bigger experiments.
When developing validphys related code interactively, be sure to read about the API functionality.
pytest: It is a framework for writing and running tests. It finds tests in the codebase (basically modules and functions that start with
test), enhances the
assertstatement to provide rich error reporting and allows to structure dependencies between the tests (in a way similar to
reportengine). Tests are stored in the codebase and executed by pytest either manually or as a part of the continuous integration process.
coverage.pyis a program that traces which lines of code have been executed when a given Python program (notably pytest) is running. The main use case is to verify that tests probe our code paths.
Code quality and reviewing
See also Reviewing pull requests. Note that these can typically be integrated with your editor of choice.
blackcode formatter runs almost without configuration and produces typically good results. It is good to run it by default, to avoid spending time on formatting (or arguing about it).
Usually the most efficient way to debug a piece of Python code, such as a
validphys action is to insert
import IPython IPython.embed()
at the location of the code you want to debug. You will then be able to query (and manipulate) the state of the code using a rich shell.
PDB: The standard Python debugger can be used as an alternative. Compared to
IPythonit has the advantage that it allows to automatically step in the execution of the code, but the disadvantage that the interface is somewhat more complex and often surprising (hint: always prefix interpreter commands with
Sometimes a piece of code runs slower than expected. The reasons can often be surprising. It is a good idea to measure where the problems actually are.
py-spy: A performance measuring program (profiler) that provides good information and little overhead. Prefer it to the standard
cProfile. The output is typically presented in the form of “Flamegraphs” that show the relative time spent on each piece of code.
We use the Sphinx tool to document code projects. It can render and organize special purpose documentation files as well as read Python source files to automatically document interfaces. It supports extensive customization and plugins. In particular because the default formatting for docstrings is somewhat unwieldy, it is recommended to enable the
napoleonextension which allows for a more lenient
numpydocstyle. Similarly the default RST markup language can be overwhelming for simple documents. We enable the recommonmark extension to be able to compose files also in markdown format.
Python static checks and code style
We use Pylint to provide static checking e.g.
finding basic errors that a compiler would catch in compiled languages. An example
is using an unknown variable name. Pylint also provides basic guidelines on the
structure of the code (e.g. avoid functions that are to complicated). Because
Pylint is way too pedantic by default, we limit the checks to only those
considered useful. The
.pylintrc file at the top level configures Pylint to
only mind those checks. Most Python IDEs and editors have some kind of support
for Pylint. It is strongly recommended to configure the editor to show the
problematic pieces of code proactively.
New code should use the Black tool to format the code. This tool should not be used to aggressively reformat existing files.
Matplotlib Image Comparison Tests
It is possible to create tests which perform an image comparison between a generated plot and a pre-existing baseline plot. Clearly this allows one to check consistency in figure generation.
Before beginning you will need to ensure that you have the tests dependencies,
which can be checked in
The next step is to write the test function. It is highly recommended to use the
validphys API for this, both to simplify the code and to make it agnostic to the
structure of backend providers - provided that they produce the same results. See
for example a function which tests the
@pytest.mark.mpl_image_compare def test_plotpdfs(): pdfs = ['NNPDF31_nnlo_as_0118'] Q = 10 flavours = ['g'] #plot_pdfs returns a generator with (figure, name_hint) return next(API.plot_pdfs(pdfs=pdfs, Q=Q, flavours=flavours))
We see that the function needs to return a valid matplotlib figure, and should
be decorated with
Now the baseline figure needs to be generated, this can be achieved by running
pytest -k <name of file containing test function> --mpl-generate-path=baseline
which will generate a PNG of the figure in the
directory. It is recommended to put all baseline plots in this directory so that
they are automatically installed, and so will be in the correct location when
the CI runs the test suite.
Now that the baseline figure exists you can check that your test works:
pytest -k <name of file containing test function> --mpl
Also you can check that the test has been added to the full test suite:
pytest --pyargs --mpl validphys
Just note that if you do not put the
--mpl flag then the test will just check
that the function runs without error, and won’t check that the output matches to