Abstract:Images from outdoor scenes may be taken under various weather conditions. It is well studied that weather impacts the performance of computer vision algorithms and needs to be handled properly. However, existing algorithms model weather condition as a discrete status and estimate it using multi-label classification. The fact is that, physically, specifically in meteorology, weather are modeled as a continuous and transitional status. Instead of directly implementing hard classification as existing multi-weather classification methods do, we consider the physical formulation of multi-weather conditions and model the impact of physical-related parameter on learning from the image appearance. In this paper, we start with solid revisit of the physics definition of weather and how it can be described as a continuous machine learning and computer vision task. Namely, we propose to model the weather uncertainty, where the level of probability and co-existence of multiple weather conditions are both considered. A Gaussian mixture model is used to encapsulate the weather uncertainty and a uncertainty-aware multi-weather learning scheme is proposed based on prior-posterior learning. A novel multi-weather co-presence estimation transformer (MeFormer) is proposed. In addition, a new multi-weather co-presence estimation (MePe) dataset, along with 14 fine-grained weather categories and 16,078 samples, is proposed to benchmark both conventional multi-label weather classification task and multi-weather co-presence estimation task. Large scale experiments show that the proposed method achieves state-of-the-art performance and substantial generalization capabilities on both the conventional multi-label weather classification task and the proposed multi-weather co-presence estimation task. Besides, modeling weather uncertainty also benefits adverse-weather semantic segmentation.

Scene simulation under different weathers

Generation of Multi-Spectral Scene Images under Different Weather Conditions

Scene Simulation for LLL Night Vision System Based on Daytime Image

A New Interactive Enhanced Method for Image with Poor Visibility

Image degradation and recovery based on multiple scattering in remote sensing and bad weather condition

Recovering of Images Degraded by Atmosphere

Image visibility enhancement based on atmospheric transmission theory and weight analysis under bad weather condition

Basis Image Decomposition for Outdoor Lighting Estimation

Simulation of infrared imaging in different atmospheric conditions

Computer Graphics in China: A practical approach for real-time illumination estimation of outdoor videos

Lighting Simulation of Augmented Outdoor Scene Based on a Legacy Photograph.

A time, space and color-based classification of different weather conditions.

The Design of a Complex Weather Environment Simulation System for Evaluating Imaging Detection Efficiency

Real-time simulation of snowing scene

A Global Infrared Image Synthesis Model for Large-Scale Complex Urban Scene

Real-Time walkthrough system of large-scale infrared dynamic scene

Light source estimation of outdoor scenes for mixed reality

Simulation of High Realism Foggy Image for Natural Scene

Real-time Snowing Simulation

Basis Image Decomposition of Outdoor Time-Lapse Videos.

Modeling Weather Uncertainty for Multi-weather Co-Presence Estimation