n3fit.hyper_optimization package


n3fit.hyper_optimization.filetrials module

Custom hyperopt trial object for persistent file storage in the form of a json file within the nnfit folder

class n3fit.hyper_optimization.filetrials.FileTrials(replica_path, parameters=None, **kwargs)[source]

Bases: Trials

Stores trial results on the fly inside the nnfit replica folder

  • replica_path (path) – Replica folder as generated by n3fit

  • parameters (dict) – Dictionary of parameters on which we are doing hyperoptimization

new_trial_docs(tids, specs, results, miscs)[source]

This is the “flushing” method which is called at the end of every trial to save things in the database. We are are overloading it in order to also write to a json file with every single trial.

n3fit.hyper_optimization.filetrials.space_eval_trial(space, trial)[source]

This function is a wrapper around hyperopt’s space eval in order to add to the json a dictionary containing the human-readable values. i.e., the standard json would say: “optimizer = [5]” and we want it to say optimizer = “Adam” But all this function does before calling hyperopt’s space_eval is to “unlist” the items. If you think space_eval should do that by itself, you are not alone https://github.com/hyperopt/hyperopt/issues/383#issuecomment-378561408

# Arguments:
  • space: the dictionary containing the hyperopt space samplers we pass

    to the hyperparametrizable function

  • trial: trial dictionary. This is a dictionary containing (among other things)

    the list of parameters that were tried for this iteration of hyperopt

# Returns:

A dictionary containing the values of all the parameters in a human-readable format

n3fit.hyper_optimization.hyper_scan module

n3fit.hyper_optimization.penalties module

Penalties that can be applied to the hyperopt loss

Penalties in this module usually take as signature the positional arguments:

pdf_models: list(n3fit.backends.keras_backend.MetaModel)

list of models or functions taking a (1, xgrid_size, 1) array as input and returns a (1, xgrid_size, 14) pdf.

stopping_object: n3fit.stopping.Stopping

object holding the information about the validation model and the stopping parameters

although not all penalties use both.

And return a float to be added to the hyperscan loss.

New penalties can be added directly in this module. The name in the runcard must match the name used in this module.

n3fit.hyper_optimization.penalties.integrability(pdf_models=None, **_kwargs)[source]

Adds a penalty proportional to the value of the integrability integration It adds a 0-penalty when the value of the integrability is equal or less than the value of the threshold defined in validphys::fitveto

The penalty increases exponentially with the growth of the integrability number


>>> from n3fit.hyper_optimization.penalties import integrability
>>> from n3fit.model_gen import pdfNN_layer_generator
>>> fake_fl = [{'fl' : i, 'largex' : [0,1], 'smallx': [1,2]} for i in ['u', 'ubar', 'd', 'dbar', 'c', 'g', 's', 'sbar']]
>>> pdf_model = pdfNN_layer_generator(nodes=[8], activations=['linear'], seed=0, flav_info=fake_fl, fitbasis="FLAVOUR")
>>> isinstance(integrability(pdf_model), float)
n3fit.hyper_optimization.penalties.patience(stopping_object=None, alpha=0.0001, **_kwargs)[source]

Adds a penalty for fits that have finished too soon, which means the number of epochs or its patience is not optimal. The penalty is proportional to the validation loss and will be 0 when the best epoch is exactly at max_epoch - patience The alpha factor is chosen so that at 10k epochs distance the penalty is 2.7 * val_loss


alpha (float) – dumping factor for the exponent


>>> from n3fit.hyper_optimization.penalties import patience
>>> from types import SimpleNamespace
>>> fake_stopping = SimpleNamespace(e_best_chi2=1000, stopping_patience=500, total_epochs=5000, vl_loss=2.42)
>>> patience(fake_stopping, alpha=1e-4)
n3fit.hyper_optimization.penalties.saturation(pdf_models=None, n=100, min_x=1e-06, max_x=0.0001, flavors=None, **_kwargs)[source]

Checks the pdf models for saturation at small x by checking the slope from min_x to max_x. Sum the saturation loss of all pdf models

  • n (int) – Number of point to evaluate the saturation

  • min_x (float) – Initial point for checking the slope

  • max_x (float) – Final point for checking the slope

  • flavors (list(int)) – indices of the flavors to inspect


>>> from n3fit.hyper_optimization.penalties import saturation
>>> from n3fit.model_gen import pdfNN_layer_generator
>>> fake_fl = [{'fl' : i, 'largex' : [0,1], 'smallx': [1,2]} for i in ['u', 'ubar', 'd', 'dbar', 'c', 'g', 's', 'sbar']]
>>> pdf_model = pdfNN_layer_generator(nodes=[8], activations=['linear'], seed=0, flav_info=fake_fl, fitbasis="FLAVOUR")
>>> isinstance(saturation(pdf_model, 5), float)

n3fit.hyper_optimization.rewards module

Target functions to minimize during hyperparameter scan

Not all functions will use all arguments. Keyword arguments that model_trainer.py will pass to this file are:

  • fold_losses: a list with the loss of each fold

  • n3pdfs: a list of N3PDF objects for each fit (which can contain more than 1 replica)

  • experimental_models: a reference to the model that contains the cv for all data (no masks)

New loss functions can be added directly in this module the name in the runcard must match the name in the module


>>> import n3fit.hyper_optimization.rewards
>>> f = ["average", "best_worst", "std"]
>>> losses = [2.34, 1.234, 3.42]
>>> for fname in f:
>>>    fun = getattr(n3fit.hyper_optimization.rewards, fname)
>>>    print(f"{fname}: {fun(losses, None):2.4f}")
average: 2.3313
best_worst: 3.4200
std: 0.8925
n3fit.hyper_optimization.rewards.average(fold_losses=None, **_kwargs)[source]

Returns the average of fold losses

n3fit.hyper_optimization.rewards.best_worst(fold_losses=None, **_kwargs)[source]

Returns the maximum loss of all k folds

n3fit.hyper_optimization.rewards.fit_distance(n3pdfs=None, **_kwargs)[source]

Loss function for hyperoptimization based on the distance of the fits of all folds to the first fold

n3fit.hyper_optimization.rewards.fit_future_tests(n3pdfs=None, experimental_models=None, **_kwargs)[source]

Use the future tests as a metric for hyperopt

NOTE: this function should only be called once at the end of every hyperopt iteration, as it is destructive for the models

n3fit.hyper_optimization.rewards.std(fold_losses=None, **_kwargs)[source]

Return the standard dev of the losses of the folds

Module contents