Generating reports

This section explains how to generate reports. These are collections of plots, tables or other validphys outputs which can be uploaded to the vp server.

  • Reports are implemented as an action of reportengine.

  • The report action takes a template argument, corresponding to the filename of a template in the Pandoc Markdown format, with the actions defined with a special syntax discussed below.

  • The actions will be resolved as if they were directly specified in the configuration file and when all of them are completed, their value will be substituted in the template (the jinja2 library is used for the intermediate rendering).

reportengine will interpret strings between {@ and @} inside the templates. There are currently target and with/endwith tags:

1. Target tags specify an action to be executed. The possible syntax is:

{@[spec] action_name[(arg1=value, arg2=value)]@}

where [] stands for optional syntax. A few conforming examples are:

{@ plot_fancy @}

{@theory::pdfs plot_fancy@}

{@plot_fancy(normalize_to=data)@}

The different parts of the specification, namely mappings, lists of mappings (or special tags implementing that behaviour) are separated with the :: operator (resembling the C++ scope resolution operator). Actions will be repeated if the specification results in multiple namespaces (e.g. one plot per pdf in the second example above).

2. With/endwith tags repeat the content between the tags for each namespace in the specifications. Targets inside the block are repeated and searched for within each namespace. The syntax of the with tag is:

{@with spec@}

and it must be closed by an endwith tag

{@endwith@}

Like in the target tag, the spec is separated by ::.

The report action

As always, see validphys –help report for the most complete information. The options allow customizing the CSS style or the template that contains the report itself.

Here we only discuss a couple of interesting flags.

  1. The main flag

The main: True flag can only affect one report per run. It has the effect of setting the name index.html, which comes in handy for visualizing the uploaded result in the server.

The main flag also tries to open the web browser when the report finishes. The browser will be chosen according to internal heuristics, by querying system preferences. These can be overridden by setting the BROWSER environment variable. For example, in text-only environments such as remote clusters, it may be preferable to just print the URL. This can be achieved by setting the environment variable to echo (for example in the .bashrc file):

export BROWSER=echo

  1. Displaying math (the mathjax flag)

Displaying math on browsers is painful and not without trouble. Pandoc tries to render the LaTeX math using utf8-characters. This does not require external dependencies and allows one to work with the text normally, but is extremely limited (little more than subindexes and greek letters).

It is possible to set mathjax: True to use the Mathjax library. This supports many more symbols, but is rather slow and requires an external connection in order to render the math.

Example report template

A template that could correspond to the example above is:

NNPDF Report
============

{@ description  @}


PDF plots
---------

{@ plot_pdfs @}

**Normalized**

{@normalize plot_pdfs  @}


Train-valid split
------------------

{@ plot_training_validation @}

$\chi^2$
-------
{@ with pdfs  @}

### {@ pdf @}

{@ experiments_chi2_table @}

{@ endwith@}

Experiment plots
---------------
{@ with pdfs @}
###Experiment results for {@pdf@}
{@with datanorm::experiments@}

#### {@experiment@}
{@experiment plot_fancy @}
{@ endwith @}
{@ endwith @}

First we are writing a verbatim Markdown title. Next we are asking for a variable named “description” to be computed and later substituted right below (it is obtained from the fit config file, as seen in the template). Then we are computing absolute and normalized PDF plots (normalize is an arbitrary string that is defined in the config file to normalize to the first PDF). We then plot the training and validation \(\chi^2\) of each replica in the fit. Next we compute the \(\chi^2\) for each experiment, and produce a separate table and heading for each PDF in pdfs (note that LaTeX math syntax is allowed). Finally we produce, for each pdf and for each experiment, a set of data-theory comparison plots (which in turn are repeated for each dataset in the experiment).

Customizing how things look in the report

By default, the str() method will be applied to objects that appear in the report. If you want a custom behaviour, declare a custom as_markdown property for your objects. It should return a string in Pandoc Markdown describing your object. Raw HTML is also allowed (although that decreases the compatibility, e.g. if we decide to output LaTeX instead of HTML in the future).