Abstract:Abstract. The current field of autonomous driving has achieved superior object detection performance in good weather conditions. However, the environment sensing capability of autonomous vehicles is severely affected in rainfall traffic environments. Although deep-learning-based image derain algorithms have made significant progress, integrating them with high-level vision tasks, such as object detection, remains challenging due to the significant differences between the derain and object detection algorithms. Additionally, the accuracy of object detection in real rain traffic environments is significantly reduced due to the domain transfer problem between the training dataset and the actual rain environment. To address this domain-shifting problem, we propose an adaptive rain image enhancement object detection network for autonomous driving in adverse weather conditions (ARODNet). This network architecture consists of an image adaptive enhancement module, an image derain module, and an object detection module. The baseline detection module (CBAM-YOLOv7) is built by incorporating the YOLOv7 object detection network into a feed-forward convolutional neural network, and it includes an attention module (CBAM). We propose a domain adaptive rain image enhancement module, DRIP, for low-quality images acquired under heavy rainfall conditions. DRIP enhances low-quality images on rainy days by adaptively learning multiple preprocessing weights. To remove the effects of rain patterns and fog clouds on image detection, we introduce DRIP-enhanced images into the depth estimation derain module (DeRain) to prevent rain and fog from obscuring the objects to be detected. Finally, the multistage joint training strategy is adopted to improve the training efficiency, and the object detection is performed while the image is derained. The efficacy of the ARODNet network for object detection in rainy weather traffic environments has been demonstrated through numerous quantitative and qualitative studies.

Image Deraining and Denoising Convolutional Neural Network For Autonomous Driving

Image Deraining and Denoising Convolutional Neural Network ForAutonomous Driving

An Interpretable Image Denoising Framework Via Dual Disentangled Representation Learning

Autonomous Driving with Human Guided Image Feature Extraction

Research on Visual Perception Technology of Autonomous Driving Based on Improved Convolutional Neural Network

An End-to-End Cascaded Image Deraining and Object Detection Neural Network

Task-Driven Deep Image Enhancement Network for Autonomous Driving in Bad Weather

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

GAN Inversion Based Point Clouds Denoising in Foggy Scenarios for Autonomous Driving

NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes

Robust autonomous driving control using deep hybrid-learning network under rainy/snown conditions

Real-Time Evaluation of Perception Uncertainty and Validity Verification of Autonomous Driving

Robust Perception under Adverse Conditions for Autonomous Driving Based on Data Augmentation

ARODNet: adaptive rain image enhancement object detection network for autonomous driving in adverse weather conditions

Learning to Remove Bad Weather: Towards Robust Visual Perception for Self-Driving

AllWeatherNet:Unified Image enhancement for autonomous driving under adverse weather and lowlight-conditions

DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving

High-Level Task-Driven Single Image Deraining: Segmentation in Rainy Days.

Single Traffic Image Deraining Via Similarity-Diversity Model

Fast Recurrent Fully Convolutional Networks for Direct Perception in Autonomous Driving

Object Detection Under Rainy Conditions for Autonomous Vehicles: A Review of State-of-the-Art and Emerging Techniques