Abstract:Effective clinical deployment of deep learning models in healthcare demands high generalization performance to ensure accurate diagnosis and treatment planning. In recent years, significant research has focused on improving the generalization of deep learning models by regularizing the sharpness of the loss landscape. Among the optimization approaches that explicitly minimize sharpness, Sharpness-Aware Minimization (SAM) has shown potential in enhancing generalization performance on general domain image datasets. This success has led to the development of several advanced sharpness-based algorithms aimed at addressing the limitations of SAM, such as Adaptive SAM, surrogate-Gap SAM, Weighted SAM, and Curvature Regularized SAM. These sharpness-based optimizers have shown improvements in model generalization compared to conventional stochastic gradient descent optimizers and their variants on general domain image datasets, but they have not been thoroughly evaluated on medical images. This work provides a review of recent sharpness-based methods for improving the generalization of deep learning networks and evaluates the methods performance on medical breast ultrasound images. Our findings indicate that the initial SAM method successfully enhances the generalization of various deep learning models. While Adaptive SAM improves generalization of convolutional neural networks, it fails to do so for vision transformers. Other sharpness-based optimizers, however, do not demonstrate consistent results. The results reveal that, contrary to findings in the non-medical domain, SAM is the only recommended sharpness-based optimizer that consistently improves generalization in medical image analysis, and further research is necessary to refine the variants of SAM to enhance generalization performance in this field

What problem does this paper attempt to address?

The paper primarily explores whether sharpness-based optimizers can improve the generalization ability of deep learning models in the field of medical image analysis. The core issue of the study is to evaluate the impact of several sharpness-based optimization methods (such as Sharpness-Aware Minimization, SAM) and their variants (including ASAM, GSAM, WSAM, and CR-SAM) on model performance on medical image datasets. Specifically, the paper focuses on the following aspects: 1. **Background and Motivation**: Effective clinical deployment requires deep learning models to have good generalization performance to ensure accurate diagnosis and treatment planning. Recent research has focused on improving the generalization ability of deep learning models by regularizing the sharpness of the loss landscape. Among these, SAM has been shown to enhance model generalization performance on general domain image datasets. However, the effectiveness of these methods on medical images has not been fully validated. 2. **Research Objectives**: This work aims to review recent sharpness-based methods and evaluate their performance on medical breast ultrasound images. The study particularly focuses on whether SAM and its variants can consistently improve model generalization performance in medical image analysis. 3. **Experimental Design**: The authors used a large publicly available breast ultrasound image dataset GDPH&SYSUCC and selected four popular models (ResNet50, VGG16, ViT, and Swin Transformer) for experiments. By comparing the test accuracy, training speed, and geometric properties of the loss landscape (such as the eigenvalues of the Hessian matrix) of models using different optimizers, the effectiveness of various optimizers was evaluated. 4. **Main Findings**: The results show that SAM is the only sharpness-based optimizer that consistently improves the generalization performance of all tested models. Additionally, the eigenvalues of the Hessian matrix indicate that the loss landscape produced by SAM is flatter than that produced by the standard Adam optimizer. Other sharpness-based optimizers such as ASAM, GSAM, WSAM, and CR-SAM did not show consistent improvement. In summary, the paper attempts to address whether sharpness-based optimization methods can effectively enhance the generalization ability of deep learning models in the field of medical image analysis and validates the effectiveness of SAM in this domain through practical testing of various optimizers.

Do Sharpness-based Optimizers Improve Generalization in Medical Image Analysis?

Fundamental Convergence Analysis of Sharpness-Aware Minimization

Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning

How Sharpness-Aware Minimization Minimizes Sharpness?

Towards Understanding Sharpness-Aware Minimization

A Method for Medical Microscopic Images’ Sharpness Evaluation Based on NSST and Variance by Combining Time and Frequency Domains

Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term

Why Does Sharpness-Aware Minimization Generalize Better Than SGD?

Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalization

The Crucial Role of Normalization in Sharpness-Aware Minimization.

Friendly Sharpness-Aware Minimization

Sharpness-Aware Gradient Matching for Domain Generalization

Critical Influence of Overparameterization on Sharpness-aware Minimization

GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization

Towards Efficient and Scalable Sharpness-Aware Minimization

Bilateral Sharpness-Aware Minimization for Flatter Minima

Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training

Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate

Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategy

A Universal Class of Sharpness-Aware Minimization Algorithms

Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach