A Foggy Weather Simulation Algorithm for Traffic Image Synthesis Based on Monocular Depth Estimation

Minan Tang,Zixin Zhao,Jiandong Qiu

DOI: https://doi.org/10.3390/s24061966

IF: 3.9

2024-03-21

Sensors

Abstract:This study addresses the ongoing challenge for learning-based methods to achieve accurate object detection in foggy conditions. In response to the scarcity of foggy traffic image datasets, we propose a foggy weather simulation algorithm based on monocular depth estimation. The algorithm involves a multi-step process: a self-supervised monocular depth estimation network generates a relative depth map and then applies dense geometric constraints for scale recovery to derive an absolute depth map. Subsequently, the visibility of the simulated image is defined to generate a transmittance map. The dark channel map is then used to distinguish sky regions and estimate atmospheric light values. Finally, the atmospheric scattering model is used to generate fog simulation images under specified visibility conditions. Experimental results show that more than 90% of fog images have AuthESI values of less than 2, which indicates that their non-structural similarity (NSS) characteristics are very close to those of natural fog. The proposed fog simulation method is able to convert clear images in natural environments, providing a solution to the problem of lack of foggy image datasets and incomplete visibility data.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

This paper attempts to address the problem of achieving accurate object detection under foggy conditions. Specifically, due to the lack of foggy traffic image datasets, existing learning-based methods perform poorly in object detection under foggy conditions. To overcome this issue, the authors propose a foggy weather simulation algorithm based on monocular depth estimation to generate synthetic foggy traffic images.

### Main Issues
1. **Challenges of Object Detection in Foggy Weather**: Under foggy conditions, color distortion and loss of detailed features in images make object detection difficult.
2. **Lack of Foggy Image Datasets**: Most existing traffic image datasets are captured in clear weather, lacking image data under foggy conditions, which limits the performance improvement of data-driven object detection models.

### Solutions
1. **Foggy Weather Simulation Algorithm**:
- **Monocular Depth Estimation**: Use a self-supervised monocular depth estimation network to generate relative depth maps and recover absolute depth maps through dense geometric constraints.
- **Visibility Definition**: Define the visibility of the simulated image based on the absolute depth map and generate a transmission map.
- **Atmospheric Light Value Estimation**: Use the dark channel prior to distinguish the sky region and estimate the atmospheric light value.
- **Atmospheric Scattering Model**: Use the atmospheric scattering model to generate foggy simulation images under specified visibility conditions.

2. **Dataset Expansion**:
- **Generate FoggyBDD100K Dataset**: Based on the BDD100K dataset, use the above method to generate 16,702 foggy images, forming the new FoggyBDD100K dataset.

### Main Contributions
1. **Proposed a New Method for Foggy Object Detection Dataset Enhancement and Simulation**, significantly improving the accuracy and generalization ability of foggy object detection models without introducing additional computational costs and complex algorithm frameworks.
2. **Created the FoggyBDD100K Dataset**, containing 16,702 foggy images, which helps the community and researchers develop and validate their data-driven object detection or defogging algorithms.
3. **Quantitatively Evaluated the Generated Foggy Simulation Images**, verifying the reliability of the foggy simulation framework.

Through these methods, the paper aims to improve the accuracy of object detection under foggy conditions, thereby reducing traffic accidents and promoting sustainable socio-economic development.