n3fit.hyper_optimization package
Submodules
n3fit.hyper_optimization.filetrials module
Custom hyperopt trial object for persistent file storage in the form of a json file within the nnfit folder
- class n3fit.hyper_optimization.filetrials.FileTrials(replica_path, parameters=None, **kwargs)[source]
Bases:
Trials
Stores trial results on the fly inside the nnfit replica folder
- Parameters:
replica_path (path) – Replica folder as generated by n3fit
parameters (dict) – Dictionary of parameters on which we are doing hyperoptimization
- n3fit.hyper_optimization.filetrials.space_eval_trial(space, trial)[source]
This function is a wrapper around hyperopt’s space eval in order to add to the json a dictionary containing the human-readable values. i.e., the standard json would say: “optimizer = [5]” and we want it to say optimizer = “Adam” But all this function does before calling hyperopt’s space_eval is to “unlist” the items. If you think space_eval should do that by itself, you are not alone https://github.com/hyperopt/hyperopt/issues/383#issuecomment-378561408
- # Arguments:
- space: the dictionary containing the hyperopt space samplers we pass
to the hyperparametrizable function
- trial: trial dictionary. This is a dictionary containing (among other things)
the list of parameters that were tried for this iteration of hyperopt
- # Returns:
A dictionary containing the values of all the parameters in a human-readable format
n3fit.hyper_optimization.hyper_scan module
n3fit.hyper_optimization.penalties module
Penalties that can be applied to the hyperopt loss
Penalties in this module usually take as signature the positional arguments:
- pdf_models: list(
n3fit.backends.keras_backend.MetaModel
)list of models or functions taking a
(1, xgrid_size, 1)
array as input and returns a(1, xgrid_size, 14)
pdf.- stopping_object:
n3fit.stopping.Stopping
object holding the information about the validation model and the stopping parameters
although not all penalties use both.
And return a float to be added to the hyperscan loss.
New penalties can be added directly in this module. The name in the runcard must match the name used in this module.
- n3fit.hyper_optimization.penalties.integrability(pdf_models=None, **_kwargs)[source]
Adds a penalty proportional to the value of the integrability integration It adds a 0-penalty when the value of the integrability is equal or less than the value of the threshold defined in validphys::fitveto
The penalty increases exponentially with the growth of the integrability number
Example
>>> from n3fit.hyper_optimization.penalties import integrability >>> from n3fit.model_gen import pdfNN_layer_generator >>> fake_fl = [{'fl' : i, 'largex' : [0,1], 'smallx': [1,2]} for i in ['u', 'ubar', 'd', 'dbar', 'c', 'g', 's', 'sbar']] >>> pdf_model = pdfNN_layer_generator(nodes=[8], activations=['linear'], seed=0, flav_info=fake_fl, fitbasis="FLAVOUR") >>> isinstance(integrability(pdf_model), float) True
- n3fit.hyper_optimization.penalties.patience(stopping_object=None, alpha=0.0001, **_kwargs)[source]
Adds a penalty for fits that have finished too soon, which means the number of epochs or its patience is not optimal. The penalty is proportional to the validation loss and will be 0 when the best epoch is exactly at max_epoch - patience The
alpha
factor is chosen so that at 10k epochs distance the penalty is 2.7 * val_loss- Parameters:
alpha (float) – dumping factor for the exponent
Example
>>> from n3fit.hyper_optimization.penalties import patience >>> from types import SimpleNamespace >>> fake_stopping = SimpleNamespace(e_best_chi2=1000, stopping_patience=500, total_epochs=5000, vl_loss=2.42) >>> patience(fake_stopping, alpha=1e-4) 3.434143467595683
- n3fit.hyper_optimization.penalties.saturation(pdf_models=None, n=100, min_x=1e-06, max_x=0.0001, flavors=None, **_kwargs)[source]
Checks the pdf models for saturation at small x by checking the slope from
min_x
tomax_x
. Sum the saturation loss of all pdf models- Parameters:
Example
>>> from n3fit.hyper_optimization.penalties import saturation >>> from n3fit.model_gen import pdfNN_layer_generator >>> fake_fl = [{'fl' : i, 'largex' : [0,1], 'smallx': [1,2]} for i in ['u', 'ubar', 'd', 'dbar', 'c', 'g', 's', 'sbar']] >>> pdf_model = pdfNN_layer_generator(nodes=[8], activations=['linear'], seed=0, flav_info=fake_fl, fitbasis="FLAVOUR") >>> isinstance(saturation(pdf_model, 5), float) True
n3fit.hyper_optimization.rewards module
Target functions to minimize during hyperparameter scan
Not all functions will use all arguments. Keyword arguments that model_trainer.py will pass to this file are:
fold_losses: a list with the loss of each fold
n3pdfs: a list of N3PDF objects for each fit (which can contain more than 1 replica)
experimental_models: a reference to the model that contains the cv for all data (no masks)
New loss functions can be added directly in this module the name in the runcard must match the name in the module
Example
>>> import n3fit.hyper_optimization.rewards
>>> f = ["average", "best_worst", "std"]
>>> losses = [2.34, 1.234, 3.42]
>>> for fname in f:
>>> fun = getattr(n3fit.hyper_optimization.rewards, fname)
>>> print(f"{fname}: {fun(losses, None):2.4f}")
average: 2.3313
best_worst: 3.4200
std: 0.8925
- n3fit.hyper_optimization.rewards.average(fold_losses=None, **_kwargs)[source]
Returns the average of fold losses
- n3fit.hyper_optimization.rewards.best_worst(fold_losses=None, **_kwargs)[source]
Returns the maximum loss of all k folds
- n3fit.hyper_optimization.rewards.fit_distance(n3pdfs=None, **_kwargs)[source]
Loss function for hyperoptimization based on the distance of the fits of all folds to the first fold