Evaluation of neural network algorithms for atmospheric turbulence mitigation

Tushar Jain,Madeline Lubien,Jerome Gilles
DOI: https://doi.org/10.1117/12.2614567
2024-10-28
Abstract:A variety of neural networks architectures are being studied to tackle blur in images and videos caused by a non-steady camera and objects being captured. In this paper, we present an overview of these existing networks and perform experiments to remove the blur caused by atmospheric turbulence. Our experiments aim to examine the reusability of existing networks and identify desirable aspects of the architecture in a system that is geared specifically towards atmospheric turbulence mitigation. We compare five different architectures, including a network trained in an end-to-end fashion, thereby removing the need for a stabilization step.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to mitigate the impact of atmospheric turbulence on images and videos by evaluating different neural network architectures. Specifically, the authors aim to: 1. **Study the applicability of existing neural network architectures**: Evaluate the effectiveness of existing neural network architectures in dealing with image blurring problems caused by atmospheric turbulence, and determine whether these architectures can be directly applied to the correction of atmospheric turbulence. 2. **Create a publicly available dataset**: To promote research in this field, the authors created a publicly available dataset named SOTIS (Simulated Open Turbulence Image Set), which contains image sequences under the influence of simulated atmospheric turbulence and their corresponding clear images (i.e., ground - truth). This helps researchers conduct quantitative evaluations and model training. 3. **Propose an evaluation method**: Given the diversity of atmospheric turbulence correction algorithms, the authors proposed a unified evaluation method to compare the performance of different algorithms in a fair and quantitative manner. They used Peak Signal - to - Noise Ratio (PSNR) and Structural Similarity Index (SSIM) as the main evaluation indicators. 4. **Explore the potential of deep learning in atmospheric turbulence correction**: By comparing classical algorithms and deep - learning - based algorithms, the authors hope to explore the performance of neural networks in this specific task and provide guidance for future research. ### Specific problem description Atmospheric turbulence can cause inhomogeneity in the refractive index in the optical path, resulting in image blurring and geometric distortion. Traditional correction methods are usually divided into two steps: - **Stabilization**: Compensate for geometric deformations in the image. - **Deblurring**: Restore the blurred image. However, due to the anisoplanatic nature of the blurring caused by atmospheric turbulence, the effectiveness of traditional methods is limited. Therefore, the authors attempt to use neural networks to solve this problem, and pay special attention to the following aspects: - **Reusability of existing neural networks**: Test whether existing neural network architectures for deblurring can be directly applied to atmospheric turbulence correction. - **End - to - end training**: Explore whether the process can be simplified by end - to - end training and the stabilization step can be removed. ### Main contributions 1. **SOTIS dataset**: Provide a large number of image sequences under the influence of simulated atmospheric turbulence and their corresponding clear images, filling the gap in the lack of publicly available datasets in this field. 2. **Systematic evaluation**: Through the systematic evaluation of multiple neural network architectures, reveal the performance differences of different architectures in the atmospheric turbulence correction task. 3. **Application prospects of deep learning**: Demonstrate the potential of deep learning in atmospheric turbulence correction and provide a basis for further research. Through these efforts, the authors hope to promote the development of atmospheric turbulence correction technology, especially to improve image quality in long - distance sensor applications.