Abstract:For the convolutional neural network (CNN) used for pattern classification, the training loss function is usually applied to the final output of the network, except for some regularization constraints on the network parameters. However, with the increasing of the number of network layers, the influence of the loss function on the network front layers gradually decreases, and the network parameters tend to fall into local optimization. At the same time, it is found that the trained network has significant information redundancy at all stages of features, which reduces the effectiveness of feature mapping at all stages and is not conducive to the change of the subsequent parameters of the network in the direction of optimality. Therefore, it is possible to obtain a more optimized solution of the network and further improve the classification accuracy of the network by designing a loss function for restraining the front stage features and eliminating the information redundancy of the front stage features .For CNN, this article proposes a multi-stage feature decorrelation loss (MFD Loss), which refines effective features and eliminates information redundancy by constraining the correlation of features at all stages. Considering that there are many layers in CNN, through experimental comparison and analysis, MFD Loss acts on multiple front layers of CNN, constrains the output features of each layer and each channel, and performs supervision training jointly with classification loss function during network training. Compared with the single Softmax Loss supervised learning, the experiments on several commonly used datasets on several typical CNNs prove that the classification performance of Softmax Loss+MFD Loss is significantly better. Meanwhile, the comparison experiments before and after the combination of MFD Loss and some other typical loss functions verify its good universality.

Improving CNN Performance Accuracies with Min-Max Objective

Improving CNN Performance with Min-Max Objective

MLCNN: Cross-Layer Cooperative Optimization and Accelerator Architecture for Speeding Up Deep Learning Applications

Improving Deep Neural Network Performance by Integrating Kernelized Min-Max Objective

MaxQ: Multi-Axis Query for N:M Sparsity Network

Improving Convolutional Neural Networks Via Compacting Features

Sensitivity-Oriented Layer-Wise Acceleration and Compression for Convolutional Neural Network.

Integrating Supervised Laplacian Objective with CNN for Object Recognition.

Improving Deep Neural Networks with Multilayer Maxout Networks

Optimization of CNN through Novel Training Strategy for Visual Classification Problems

Optimizing Convolutional Neural Network Architecture

Accelerating Convolutional Neural Networks Using Fine-Tuned Backpropagation Progress

Optimizing Convolutional Neural Network Performance by Mitigating Underfitting and Overfitting.

Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization

Optimal Hyperparameters and Structure Setting of Multi-Objective Robust CNN Systems via Generalized Taguchi Method and Objective Vector Norm

Accelerating Convolutional Neural Networks by Removing Interspatial and Interkernel Redundancies.

The Investigation of Multiple Optimization Methods on Convolutional Neural Network

Two-Stage Model Compression and Acceleration: Optimal Student Network for Better Performance

Learning Efficient Convolutional Networks Through Network Slimming.

Multi-stage feature decorrelation constraints for improving CNN classification performance

Optimizing Convolutional Neural Network Architectures