n3fit.hyper_optimization package

Submodules

n3fit.hyper_optimization.filetrials module

Custom hyperopt trial object for persistent file storage in the form of a json file within the nnfit folder

class n3fit.hyper_optimization.filetrials.FileTrials(replica_path, parameters=None, **kwargs)

Bases: Trials

Stores trial results on the fly inside the nnfit replica folder

Parameters:

• replica_path (path) – Replica folder as generated by n3fit

• parameters (dict) – Dictionary of parameters on which we are doing hyperoptimization

new_trial_docs(tids, specs, results, miscs)

new_trial_ids(n)

refresh()

This is the “flushing” method which is called at the end of every trial to save things in the database. We are are overloading it in order to also write to a json file with every single trial.

n3fit.hyper_optimization.filetrials.space_eval_trial(space, trial)

This function is a wrapper around hyperopt’s space eval in order to add to the json a dictionary containing the human-readable values. i.e., the standard json would say: “optimizer = [5]” and we want it to say optimizer = “Adam” But all this function does before calling hyperopt’s space_eval is to “unlist” the items. If you think space_eval should do that by itself, you are not alone https://github.com/hyperopt/hyperopt/issues/383#issuecomment-378561408

# Arguments:

• space: the dictionary containing the hyperopt space samplers we pass

  to the hyperparametrizable function

• trial: trial dictionary. This is a dictionary containing (among other things)

  the list of parameters that were tried for this iteration of hyperopt

# Returns:

A dictionary containing the values of all the parameters in a human-readable format

n3fit.hyper_optimization.hyper_scan module

n3fit.hyper_optimization.penalties module

Penalties that can be applied to the hyperopt loss

Penalties in this module usually take as signature the positional arguments:

pdf_models: list(n3fit.backends.keras_backend.MetaModel)

list of models or functions taking a (1, xgrid_size, 1) array as input and returns a (1, xgrid_size, 14) pdf.

stopping_object: n3fit.stopping.Stopping

object holding the information about the validation model and the stopping parameters

although not all penalties use both.

And return a float to be added to the hyperscan loss.

New penalties can be added directly in this module. The name in the runcard must match the name used in this module.

n3fit.hyper_optimization.penalties.integrability(pdf_models=None, **_kwargs)

Adds a penalty proportional to the value of the integrability integration It adds a 0-penalty when the value of the integrability is equal or less than the value of the threshold defined in validphys::fitveto

The penalty increases exponentially with the growth of the integrability number

Example

>>> from n3fit.hyper_optimization.penalties import integrability
>>> from n3fit.model_gen import pdfNN_layer_generator
>>> fake_fl = [{'fl' : i, 'largex' : [0,1], 'smallx': [1,2]} for i in ['u', 'ubar', 'd', 'dbar', 'c', 'g', 's', 'sbar']]
>>> pdf_model = pdfNN_layer_generator(nodes=[8], activations=['linear'], seed=0, flav_info=fake_fl, fitbasis="FLAVOUR")
>>> isinstance(integrability(pdf_model), float)
True

n3fit.hyper_optimization.penalties.patience(stopping_object=None, alpha=0.0001, **_kwargs)

Adds a penalty for fits that have finished too soon, which means the number of epochs or its patience is not optimal. The penalty is proportional to the validation loss and will be 0 when the best epoch is exactly at max_epoch - patience The alpha factor is chosen so that at 10k epochs distance the penalty is 2.7 * val_loss

Parameters:

alpha (float) – dumping factor for the exponent

Example

>>> from n3fit.hyper_optimization.penalties import patience
>>> from types import SimpleNamespace
>>> fake_stopping = SimpleNamespace(e_best_chi2=1000, stopping_patience=500, total_epochs=5000, vl_loss=2.42)
>>> patience(fake_stopping, alpha=1e-4)
3.434143467595683

n3fit.hyper_optimization.penalties.saturation(pdf_models=None, n=100, min_x=1e-06, max_x=0.0001, flavors=None, **_kwargs)

Checks the pdf models for saturation at small x by checking the slope from min_x to max_x. Sum the saturation loss of all pdf models

Parameters:

• n (int) – Number of point to evaluate the saturation

• min_x (float) – Initial point for checking the slope

• max_x (float) – Final point for checking the slope

• flavors (list(int)) – indices of the flavors to inspect

Example

>>> from n3fit.hyper_optimization.penalties import saturation
>>> from n3fit.model_gen import pdfNN_layer_generator
>>> fake_fl = [{'fl' : i, 'largex' : [0,1], 'smallx': [1,2]} for i in ['u', 'ubar', 'd', 'dbar', 'c', 'g', 's', 'sbar']]
>>> pdf_model = pdfNN_layer_generator(nodes=[8], activations=['linear'], seed=0, flav_info=fake_fl, fitbasis="FLAVOUR")
>>> isinstance(saturation(pdf_model, 5), float)
True

n3fit.hyper_optimization.rewards module

Target functions to minimize during hyperparameter scan

Not all functions will use all arguments. Keyword arguments that model_trainer.py will pass to this file are:

• fold_losses: a list with the loss of each fold

• n3pdfs: a list of N3PDF objects for each fit (which can contain more than 1 replica)

• experimental_models: a reference to the model that contains the cv for all data (no masks)

New loss functions can be added directly in this module the name in the runcard must match the name in the module

Example

>>> import n3fit.hyper_optimization.rewards
>>> f = ["average", "best_worst", "std"]
>>> losses = [2.34, 1.234, 3.42]
>>> for fname in f:
>>>    fun = getattr(n3fit.hyper_optimization.rewards, fname)
>>>    print(f"{fname}: {fun(losses, None):2.4f}")
average: 2.3313
best_worst: 3.4200
std: 0.8925

n3fit.hyper_optimization.rewards.average(fold_losses=None, **_kwargs)

Returns the average of fold losses

n3fit.hyper_optimization.rewards.best_worst(fold_losses=None, **_kwargs)

Returns the maximum loss of all k folds

n3fit.hyper_optimization.rewards.fit_distance(n3pdfs=None, **_kwargs)

Loss function for hyperoptimization based on the distance of the fits of all folds to the first fold

n3fit.hyper_optimization.rewards.fit_future_tests(n3pdfs=None, experimental_models=None, **_kwargs)

Use the future tests as a metric for hyperopt

NOTE: this function should only be called once at the end of every hyperopt iteration, as it is destructive for the models

n3fit.hyper_optimization.rewards.std(fold_losses=None, **_kwargs)

Return the standard dev of the losses of the folds

Module contents