Regularization

Technique What? How? Links
L1 Minimize weight values from keras import regularizers http
L2 Minimize weight values. Bigger Values are heavier penalized compared to L1 from keras import regularizers http
Dropout Drop random neurons from a layer. Information needs to be encoded in multiple pathways. from keras.layers import Dropout http
Early-Stopping Stops the learning process if the score on the validation set stops improving. from keras.callbacks import EarlyStopping http
Limit layer size Reduce the complexity of the model to force abstraction of information. Reduce model size http
Data augmentation Augment the data to have a broader statistical variance and/or more data. E.g. adding noise, rotating images, handle punctuation, segmentation, annotate data, …