AllWeatherNet:Unified Image enhancement for autonomous driving under adverse weather and lowlight-conditions

Chenghao Qian,Mahdi Rezaei,Saeed Anwar,Wenjing Li,Tanveer Hussain,Mohsen Azarmi,Wei Wang

2024-09-04

Abstract:Adverse conditions like snow, rain, nighttime, and fog, pose challenges for autonomous driving perception systems. Existing methods have limited effectiveness in improving essential computer vision tasks, such as semantic segmentation, and often focus on only one specific condition, such as removing rain or translating nighttime images into daytime ones. To address these limitations, we propose a method to improve the visual quality and clarity degraded by such adverse conditions. Our method, AllWeather-Net, utilizes a novel hierarchical architecture to enhance images across all adverse conditions. This architecture incorporates information at three semantic levels: scene, object, and texture, by discriminating patches at each level. Furthermore, we introduce a Scaled Illumination-aware Attention Mechanism (SIAM) that guides the learning towards road elements critical for autonomous driving perception. SIAM exhibits robustness, remaining unaffected by changes in weather conditions or environmental scenes. AllWeather-Net effectively transforms images into normal weather and daytime scenes, demonstrating superior image enhancement results and subsequently enhancing the performance of semantic segmentation, with up to a 5.3% improvement in mIoU in the trained domain. We also show our model's generalization ability by applying it to unseen domains without re-training, achieving up to 3.9% mIoU improvement. Code can be accessed at: <a class="link-external link-https" href="https://github.com/Jumponthemoon/AllWeatherNet" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the issue of image quality degradation in autonomous driving perception systems under adverse weather conditions (such as snow, rain, night, fog) and low light conditions. Specifically, existing methods have limited effectiveness in improving key computer vision tasks (e.g., semantic segmentation) and often focus only on specific conditions (such as de-raining or night-to-day conversion). To tackle these issues, the authors propose a unified approach—AllWeather-Net, which can enhance the visual quality and clarity of images under various adverse conditions.

The main contributions include:
1. **Unified Framework**: For the first time, a unified image enhancement method is introduced to handle image quality degradation under adverse weather and low light conditions.
2. **Illumination-Aware Attention Mechanism**: A scaled illumination-aware attention mechanism (SIAM) is proposed, enabling the network to balance attention to road elements according to different weather conditions and scene changes.
3. **Hierarchical Discriminative Framework**: A novel architecture is designed to perform discriminative tasks at three semantic levels (scene, object, texture), ensuring overall image consistency while enhancing image details.

Through these technical means, AllWeather-Net can transform images under different adverse conditions into normal weather and daytime scenes, significantly improving image enhancement effects and thereby enhancing semantic segmentation performance. Experimental results show that this method outperforms existing methods on multiple datasets and has good generalization ability.