Hyperparameter search is the process of finding the optimal topology and configuration for a neural network to be trained. An optimal configuration of the hyperparameter is crucial for the resulting approximation quality. With DFFN, two strategies are provided to support an automatic identification of the optimal hyperparameter configuration, namely smart layout and optimization layout.
This comprises the identification of the following Hyperparameters:
- Learning rate
Controls how much to change the model in response to the estimated error each time the model weights are updated.
- Number of Hidden Layers and Units
Determines the architecture of the neural network.
- Activation Functions
Functions that determine the output of a neural network node.
- Regularization coefficient
A parameter
that controls the amount of regularization applied
to the model to prevent over-fitting. In the L1 regularization
technique used in the Deep Feed Forward Network, the absolute value
of weights is added to the loss function: where
is the original loss, 
is the regularization coefficient, and
are the weights. It can lead to sparse models
where individual weights will be set to zero.
In the manual layout, all Hyperparameters are to be configured by the user, requiring expert knowledge on the training of artificial neural networks.