Abstract:In recent years, convolutional neural networks (CNNs) have been used in many fields. Nowadays, CNNs have a high learning capability, and this learning capability is accompanied by a more complex model architecture. Complex model architectures allow CNNs to learn more data features, but such a learning process tends to reduce the training model's ability to generalize to unknown data, and may be associated with problems of overfitting. Although many regularization methods have been proposed, such as data augmentation, batch normalization, and Dropout, research on improving generalization performance is still a common concern in the training process of robust CNNs. In this paper, we propose a dynamically controllable adjustment method, which we call LossDA, that embeds a disturbance variable in the fully-connected layer. The trend of this variable is kept consistent with the training loss, while the magnitude of the variable can be preset to adapt to the training process of different models. Through this dynamic adjustment, the training process of CNNs can be adaptively adjusted. The whole regularization process can improve the generalization performance of CNNs while helping to suppress overfitting. To evaluate this method, this paper conducts comparative experiments on MNIST, FashionMNIST, CIFAR-10, Cats_vs_Dogs, and miniImagenet datasets. The experimental results show that the method can improve the model performance of Light CNNs and Transfer CNNs (InceptionResNet, VGG19, ResNet50, and InceptionV3). The average maximum improvement in accuracy of Light CNNs is 4.62%, F1 is 3.99%, and Recall is 4.69%. The average maximum improvement accuracy of Transfer CNNs is 4.17%, F1 is 5.64%, and Recall is 4.05%.

Dynamic Attention Loss for Small-Sample Image Classification

Inter-Class Angular Loss for Convolutional Neural Networks.

FICAL: Focal Inter-Class Angular Loss for Image Classification

Learning Representative Features Via Constrictive Annular Loss for Image Classification

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

Angular Joint Loss for Image Classification

DRFL: Dynamic-Recall Focal Loss for Image Classification and Segmentation

Low-Sample Image Classification Based on Intrinsic Consistency Loss and Uncertainty Weighting Method

Dense Attention Convolutional Network for Image Classification

Improved generalization performance of convolutional neural networks with LossDA

Dynamics-Aware Loss for Learning with Label Noise

Exploiting Web Images for Fine-Grained Visual Recognition Via Dynamic Loss Correction and Global Sample Selection

Attention-Aware Noisy Label Learning for Image Classification

CAM-loss: Towards Learning Spatially Discriminative Feature Representations

A New Loss Function for CNN Classifier Based on Predefined Evenly-Distributed Class Centroids

Attentiondrop For Convolutional Neural Networks

Attention Graph: Learning Effective Visual Features for Large-Scale Image Classification

Learning from Noisy Labels Via Dynamic Loss Thresholding

Self-Attention Network with Joint Loss for Remote Sensing Image Scene Classification

Confusion-Aware Convolutional Neural Network For Image Classification