validphys.theorycovariance package
Submodules
validphys.theorycovariance.construction module
construction.py Tools for constructing theory covariance matrices and computing their chi2s.
- class validphys.theorycovariance.construction.ProcessInfo(preds, namelist, sizes)
Bases:
tuple
- namelist
Alias for field number 1
- preds
Alias for field number 0
- sizes
Alias for field number 2
- validphys.theorycovariance.construction.combine_by_type(each_dataset_results_central_bytheory)[source]
Groups the datasets bu process and returns an instance of the ProcessInfo class
- Parameters
each_dataset_results_central_bytheory (list[list[(DataResult,ThPredictionsResult)]]) – Tuples of DataResult and ThPredictionsResult (where only the second is used for the construction of the theory covariance matrix), wrapped in a list such that there is a tuple per theoryid, wrapped in another list per dataset.
- Returns
Class with info needed to construct the theory covmat.
- Return type
ProcesInfo
validphys.theorycovariance.construction.ProcessInfo
- validphys.theorycovariance.construction.compute_covs_pt_prescrip(point_prescription, l, name1, deltas1, name2=None, deltas2=None, fivetheories=None, seventheories=None)[source]
Utility to compute the covariance matrix by prescription given the shifts with respect to the central value for a pair of processes.
The processes are defined by the variables
name1
andname2
withdeltas1
anddeltas2
the associated shifts wrt the central prediction.This utility also allows for the computation of the theory covmat for a single process or dataset if
names2=deltas2=None
.- Parameters
point_prescription (str) – defines the point prescription to be utilized
l (int) – Number of theory variations (counting the central theory)
name1 (str) – Process name of the first set of shifts
deltas1 (list(np.ndarray)) – list of shifts for each of the non-central theories
name2 (str) – Process name of the second set of shifts
deltas2 (list(np.ndarray)) – list of shifts for each of the non-central theories
fivetheories (str) – 5-point prescription variation
seventheories (str) – 7-point prescription variation
- validphys.theorycovariance.construction.covmat_3fpt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 3pt factorisation scale variation only, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_3pt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 3pt prescription, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_3rpt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 3pt renormalisation scale variation only, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_5barpt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 5barpt prescription, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_5pt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 5pt prescription, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_7pt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 7pt prescription (Gavin), given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_7pt_orig(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for original 7pt prescription, now deprecated but kept for posterity, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_9pt(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for 9pt prescription, given two dataset names and collections of scale variation shifts
- validphys.theorycovariance.construction.covmat_n3lo_ad(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for each of the singlet splitting function variations.
- Normalization is given by:
(n_pt - 1)
- where:
n_pt = number of point presctiption
- validphys.theorycovariance.construction.covmat_n3lo_fhmv(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for all the FHMV splitting function variations.
- validphys.theorycovariance.construction.covmat_n3lo_singlet(name1, name2, deltas1, deltas2)[source]
Returns theory covariance sub-matrix for all the singlet splitting function variations.
- validphys.theorycovariance.construction.covs_pt_prescrip(combine_by_type, theoryids, point_prescription, fivetheories, seventheories)[source]
Produces the sub-matrices of the theory covariance matrix according to a point prescription which matches the number of input theories. If 5 theories are provided, a scheme ‘bar’ or ‘nobar’ must be chosen in the runcard in order to specify the prescription. Sub-matrices correspond to applying the scale variation prescription to each pair of processes in turn, using a different procedure for the case where the processes are the same relative to when they are different.
- validphys.theorycovariance.construction.experimentplustheory_corrmat_custom(procs_covmat, theory_covmat_custom)[source]
Calculates the correlation matrix for the experimental plus theory covariance matrices, correlations by prescription.
- validphys.theorycovariance.construction.fromfile_covmat(covmatpath, procs_data, procs_index)[source]
Reads a general theory covariance matrix from file. Then 1: Applies cuts to match experiment covariance matrix 2: Expands dimensions to match experiment covariance matrix
by filling additional entries with 0.
- validphys.theorycovariance.construction.procs_index_matched(groups_index, procs_index)[source]
procs_index but matched to the dataset order given by groups_index.
- validphys.theorycovariance.construction.theory_corrmat_custom(theory_covmat_custom)[source]
Calculates the theory correlation matrix for scale variations with variations by process type
- validphys.theorycovariance.construction.theory_covmat_custom(covs_pt_prescrip, procs_index, combine_by_type)[source]
Takes the individual sub-covmats between each two processes and assembles them into a full covmat. Then reshuffles the order from ordering by process to ordering by experiment as listed in the runcard
- validphys.theorycovariance.construction.theory_covmat_custom_fitting(theory_covmat_custom, procs_index_matched)[source]
theory_covmat_custom but reindexed so the order of the datasets matches those in the experiment covmat so they are aligned when fitting.
- validphys.theorycovariance.construction.theory_covmat_dataset(results, results_central_bytheoryids, use_theorycovmat, point_prescription, fivetheories=None, seventheories=None)[source]
Compute the theory covmat for a collection of theoryids for a single dataset.
In general this will come from some point prescription and it could be guessed from the input however, it takes as input all relevant variables for generality
- validphys.theorycovariance.construction.theory_normcovmat_custom(theory_covmat_custom, procs_data_values)[source]
Calculates the theory covariance matrix for scale variations normalised to data, with variations according to the relevant prescription.
- validphys.theorycovariance.construction.total_theory_covmat(theory_covmat_custom, user_covmat)[source]
Sum of scale variation and user covmat, where both are used.
- validphys.theorycovariance.construction.total_theory_covmat_fitting(total_theory_covmat, procs_index_matched)[source]
total_theory_covmat but reindexed so the order of the datasets matches those in the experiment covmat so they are aligned when fitting.
- validphys.theorycovariance.construction.user_covmat(procs_data, procs_index, loaded_user_covmat_path)[source]
General theory covariance matrix provided by the user. Useful for testing the impact of externally produced covariance matrices. Matrices must be produced as a csv of pandas DataFrame, and uploaded to the validphys server. The server path is then provided via
user_covmat_path
intheorycovmatconfig
in the runcard. For more information see documentation.
validphys.theorycovariance.output module
output.py Basic tools for plotting theory covariance matrices and their properties.
- validphys.theorycovariance.output.matrix_plot_labels(df)[source]
Returns the tick locations and labels, and the starting point values for each category, based on a dataframe to be plotted. The dataframe is assumed to be multiindexed by (process, dataset, points) or else (dataset, points). The tick location is in the centre of the dataset, and labelling is by the outermost index of the multiindex.
- validphys.theorycovariance.output.plot_corrmat_heatmap(corrmat, title)[source]
Matrix plot of a correlation matrix
- validphys.theorycovariance.output.plot_covmat_heatmap(covmat, title)[source]
Matrix plot of a covariance matrix.
- validphys.theorycovariance.output.plot_diag_cov_comparison(theory_covmat_custom, procs_covmat, procs_data_values, theoryids, fivetheories)[source]
Plot of sqrt(cov_ii)/|data_i| for cov = exp, theory, exp+theory
- validphys.theorycovariance.output.plot_diag_cov_comparison_by_experiment(theory_covmat_custom_dataspecs, experiments_covmat_no_table, procs_data_values_experiment, dataspecs)[source]
Plot of sqrt(cov_ii)/|data_i| for cov = exp, theory, exp+theory, by experiment
- validphys.theorycovariance.output.plot_diag_cov_comparison_by_process(theory_covmat_custom_dataspecs, procs_covmat, procs_data_values, dataspecs)[source]
Plot of sqrt(cov_ii)/|data_i| for cov = exp, theory, exp+theory, by process
dataspecs here is used also to specify whether to plot or not the total uncertainty, defined as the experimental uncertainty plus the uncertainty coming from the theory covariance matrix for the relevant dataspec entry.
- validphys.theorycovariance.output.plot_expcorrmat_heatmap(procs_corrmat)[source]
Matrix plot of the experiment correlation matrix
- validphys.theorycovariance.output.plot_expplusthcorrmat_heatmap_custom(experimentplustheory_corrmat_custom, theoryids, fivetheories)[source]
Matrix plot of the exp + theory correlation matrix
- validphys.theorycovariance.output.plot_normexpcovmat_heatmap(procs_normcovmat)[source]
Matrix plot of the experiment covariance matrix normalised to data.
validphys.theorycovariance.tests module
tests.py Tools for testing theory covariance matrices and their properties.
- class validphys.theorycovariance.tests.LabeledShifts(process, dataset_name, shifts)
Bases:
tuple
- dataset_name
Alias for field number 1
- process
Alias for field number 0
- shifts
Alias for field number 2
- validphys.theorycovariance.tests.alltheory_vector(dataset_alltheory)[source]
Returns a DataFrame with the theory vectors for matched dataspecs for the scale-varied theories, including the central
- validphys.theorycovariance.tests.concatenated_shx_vector(shx_vector)[source]
Single DataFrame for all the datasets of the shift vectors.
- validphys.theorycovariance.tests.dataset_alltheory(each_dataset_results_central_bytheory)[source]
Returns a LabeledShifts tuple corresponding to the theory vectors for all the scale varied theories, including the central.
- validphys.theorycovariance.tests.deltamiss_plot(theory_shift_test, ordered_alltheory_vector, fnorm_shifts_ordered, tripleindex_thcovmat_complete, dataspecs)[source]
Produces a plot of the missing component of the shift vector, transformed back to the data space.
- validphys.theorycovariance.tests.diagdf_theory_covmat(theory_covmat_custom)[source]
Return a Dataframe indexed with groups and dataset of the diagonal entry of the theory covmat.
- validphys.theorycovariance.tests.doubleindex_set_byprocess(group_dataset_inputs_by_process)[source]
The (group, dataset) index ordered by process.
- validphys.theorycovariance.tests.doubleindex_thcovmat(theory_covmat_custom)[source]
The unique set of (group, dataset) index of the theory covmat.
- validphys.theorycovariance.tests.efficiency(theory_shift_test)[source]
Returns (efficiency = 1 - fmiss/f) with which the theory covariance matrix encapsulates the NNLO-NLO shift.
- validphys.theorycovariance.tests.eigenvector_plot(evals_nonzero_basis, fnorm_shifts_ordered, tripleindex_thcovmat_complete, dataspecs)[source]
Produces a plot of the eigenvectors for the projected matrix, transformed back to the data space.
- validphys.theorycovariance.tests.evals_nonzero_basis(ordered_alltheory_vector, theory_covmat_custom, fivetheories, orthonormalisation: (<class 'str'>, <class 'NoneType'>) = None, seventheories: (<class 'str'>, <class 'NoneType'>) = None)[source]
- validphys.theorycovariance.tests.fnorm_shifts_byprocess(concatenated_shx_vector, doubleindex_set_byprocess)[source]
Shift vector ordered by process.
- validphys.theorycovariance.tests.fnorm_shifts_ordered(concatenated_shx_vector, doubleindex_thcovmat)[source]
Shifts vectors ordered as the theory covmat.
- validphys.theorycovariance.tests.ordered_alltheory_vector(alltheory_vector, doubleindex_thcovmat)[source]
Order the group and the dataset of all the theory vectors of the scale-varied predictions in the same way the theory covmat is ordered.
- validphys.theorycovariance.tests.projector_eigenvalue_ratio(theory_shift_test)[source]
Produces a plot of the ratio between the projectors and the square roots of the corresponding eigenvalues.
- validphys.theorycovariance.tests.shift_diag_cov_comparison(sqrtdiags_thcovmat_byprocess, fnorm_shifts_byprocess, point_prescription, ticklocs_thcovmat, dataspecs)[source]
Plot of the comparison of a shift between two pertubative order and the diagonal entries of the theory covmat, both normalized to the first of the two perturbative orders.
- validphys.theorycovariance.tests.shift_vector(matched_dataspecs_results, process, dataset_name)[source]
Returns a DataFrame of normalised shift vectors and normalization vector for matched dataspecs.
- validphys.theorycovariance.tests.sqrtdiags_thcovmat_byprocess(doubleindex_set_byprocess, diagdf_theory_covmat, concatenated_shx_vector)[source]
Ratio of the sqrts of the diagonal entries of the theory covmat to the normalization entries of the shift vectors, ordered by process.
- validphys.theorycovariance.tests.theory_covmat_eigenvalues(theory_shift_test)[source]
Returns a table of s = sqrt(eigenvalue), the projector and the ratio of the two, ordered by largest eigenvalue.
- validphys.theorycovariance.tests.theory_shift_test(fnorm_shifts_ordered, evals_nonzero_basis)[source]
Compares the NNLO-NLO shift, f, with the eigenvectors and eigenvalues of the theory covariance matrix, and returns the component of the NNLO-NLO shift space which is missed by the covariance matrix space: fmiss, as well as the projections of the shift vector onto each of the eigenvectors: projectors.
- validphys.theorycovariance.tests.theta(theory_shift_test)[source]
Returns the angle between the NNLO-NLO shift vector and the component of this which is captured by the theory covariance matrix
- validphys.theorycovariance.tests.tripleindex_thcovmat_complete(theory_covmat_custom)[source]
The complete tripleindex of the theory covmat.
- validphys.theorycovariance.tests.validation_theory_chi2(theory_shift_test)[source]
Returns the theory chi2 for comparing NNLO-NLO shift with theory covariance matrix.
- validphys.theorycovariance.tests.vectors_3pt(splitdiffs)[source]
Returns the linearly independent vectors for 3pt prescription
- validphys.theorycovariance.tests.vectors_5barpt(splitdiffs)[source]
Returns the linearly independent vectors for 5barpt prescription
- validphys.theorycovariance.tests.vectors_5pt(splitdiffs)[source]
Returns the linearly independent vectors for 5pt prescription
validphys.theorycovariance.theorycovarianceutils module
theorycovarianceutils.py
Low level utilities for theorycovariance module
- validphys.theorycovariance.theorycovarianceutils.check_correct_theory_combination(theoryids, fivetheories, point_prescription: (<class 'str'>, <class 'NoneType'>) = None)
Checks that a valid theory combination corresponding to an existing prescription has been inputted
- validphys.theorycovariance.theorycovarianceutils.check_correct_theory_combination_internal(theoryids, fivetheories, point_prescription: (<class 'str'>, <class 'NoneType'>) = None)[source]
Checks that a valid theory combination corresponding to an existing prescription has been inputted
- validphys.theorycovariance.theorycovarianceutils.check_fit_dataset_order_matches_grouped(group_dataset_inputs_by_metadata, data_input, processed_metadata_group)[source]
Check for use with theory covmat generation.
Makes sure that the order of datasets listed in the fit runcard is the same as that specified by the metadata grouping. Otherwise there can be a misalignment between the experiment covmat and theory covmat.
Module contents
theorycovariance
Tools for constructing and studying theory covariance matrices.
This package contains tools for constructing covariance matrices and computing their chi2s in construction.py, tools for plotting covariance matrices and tabulating their properties in output.py, and tools for testing their properties in tests.py.