Comparison of Cross-Entropy, Dice, and Focal Loss for Sea Ice Type Segmentation

Rafael Pires de Lima,Behzad Vahedi,Morteza Karimzadeh
DOI: https://doi.org/10.1109/IGARSS52108.2023.10282060
2023-10-26
Abstract:Up-to-date sea ice charts are crucial for safer navigation in ice-infested waters. Recently, Convolutional Neural Network (CNN) models show the potential to accelerate the generation of ice maps for large regions. However, results from CNN models still need to undergo scrutiny as higher metrics performance not always translate to adequate outputs. Sea ice type classes are imbalanced, requiring special treatment during training. We evaluate how three different loss functions, some developed for imbalanced class problems, affect the performance of CNN models trained to predict the dominant ice type in Sentinel-1 images. Despite the fact that Dice and Focal loss produce higher metrics, results from cross-entropy seem generally more physically consistent.
Computer Vision and Pattern Recognition,Image and Video Processing
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: How to improve the performance of the Convolutional Neural Network (CNN) model in the sea - ice type segmentation task by using different loss functions (cross - entropy loss, Dice loss, and Focal loss), especially the effect when dealing with the class - imbalance problem. Specifically, the researchers hope to find a method that can generate more accurate and physically reasonable sea - ice type predictions by evaluating the influence of these loss functions on the CNN model. ### Background and Problem Description 1. **Importance of Sea - Ice Charts** - The latest sea - ice charts are crucial for safe navigation in icy waters. - In recent years, Convolutional Neural Network (CNN) models have shown the potential to accelerate the generation of large - area ice maps. 2. **Limitations of Existing Methods** - The results of CNN models still need to be carefully examined because higher performance metrics do not always translate into appropriate outputs. - The classes of sea - ice types are imbalanced and require special treatment during the training process. 3. **Research Objectives** - Evaluate the influence of three different loss functions (cross - entropy loss, Dice loss, and Focal loss) on the performance of the CNN model. - Study the performance of these loss functions when dealing with the class - imbalance problem to improve the accuracy of the model in predicting the main sea - ice types. ### Specific Problems - **Class - Imbalance Problem**: The class distribution of sea - ice types is unbalanced, and some types of sea - ice samples are fewer, resulting in difficulty for the model to correctly identify these minority classes during training. - **Physical Consistency**: Although some loss functions may perform better in terms of performance metrics, whether their prediction results are physically reasonable is also an important consideration. ### Solutions - **Experimental Design**: Conduct a series of experiments using the ExtremeEarth V2 dataset to compare the influence of different loss functions on model performance. - **Evaluation Criteria**: Not only rely on performance metrics (such as the weighted F1 - score), but also evaluate the physical reasonableness of the prediction results through visual inspection. ### Conclusions Although Dice and Focal losses perform better in some performance metrics, it is found through visual inspection that the prediction results generated by cross - entropy loss are usually more physically reasonable. This indicates that when choosing a loss function, in addition to considering performance metrics, attention also needs to be paid to the practical application effects of the prediction results. ### Formula Representation - **Cross - Entropy Loss** \[ L_{CE}=-\sum_{i = 1}^{N}y_i\log(p_i) \] where \(y_i\) is the true label and \(p_i\) is the predicted probability. - **Dice Loss** \[ L_{Dice}=1-\frac{2\sum_{i = 1}^{N}p_iy_i+\epsilon}{\sum_{i = 1}^{N}p_i^2+\sum_{i = 1}^{N}y_i^2+\epsilon} \] where \(\epsilon\) is a small constant used to prevent division - by - zero errors. - **Focal Loss** \[ L_{Focal}=-\alpha(1 - p_t)^{\gamma}\log(p_t) \] where \(\alpha\) is the balancing factor, \(\gamma\) is the modulating factor, and \(p_t\) is the predicted probability. Through these formulas, the researchers can quantify the influence of different loss functions on model training and select the most suitable loss function for the sea - ice type segmentation task.