Abstract:Transactions of the Institute of Measurement and Control, Ahead of Print. One significant challenge in improving autonomous driving algorithms is the lack of diverse real-world data. Moreover, transferring models from simulation to reality faces the reality gap problem. This study addresses this issue by developing an augmentation technique for mixed-reality environments, aimed at improving the testing and training of autonomous vehicles. Tested offline, it lays the groundwork for future online applications. The methodology focuses on creating virtual depth images using a virtual camera and applying an augmentation strategy to the KITTI data set. This creates a mixed-reality representation by combining virtual and real depth maps, leveraging depth information in the fusion process. The outcomes of this process are notably effective, achieving a balance between virtual and real-world aspects. This fusion method adeptly combines elements from both environments, maintaining the quality of the images. The novel contributions of this work include a detailed augmentation strategy that seamlessly integrates virtual objects into real depth maps, accounting for occlusions and ensuring realistic depth representations. In addition, this work demonstrates the feasibility of generating a large data set using the proposed method, significantly expanding the available data for training autonomous driving models. The use of metrics such as SSIM, peak signal-to-noise ratio (PSNR), and MAE, alongside object detection models such as Faster RCNN, provides a complete evaluation of both quantitative and qualitative aspects. The results demonstrate the quality of the augmented images, setting a foundation for potential online applications. The proposed strategy enables the generation of larger data set and facilitates safe, effective training in scenarios considered too risky or challenging to simulate accurately.

Perception Quality Evaluation with Visual and Infrared Cameras in Challenging Environmental Conditions

An Efficient Image Quality Assessment Guidance Method for Unmanned Aerial Vehicle

Environmental Perception and Sensor Data Fusion for Unmanned Ground Vehicle

Visual-Thermal Landmarks and Inertial Fusion for Navigation in Degraded Visual Environments

A Quality Index Metric and Method for Online Self-Assessment of Autonomous Vehicles Sensory Perception

Framework for Quality Evaluation of Smart Roadside Infrastructure Sensors for Automated Driving Applications

Emergent Visual Sensors for Autonomous Vehicles

Automated Evaluation of Semantic Segmentation Robustness for Autonomous Driving

Analysis of Thermal Imaging Performance under Extreme Foggy Conditions: Applications to Autonomous Driving

Quality Assessment for Visible and Infrared Color Fusion Images of Typical Scenes

UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment

Experimental Evaluation of Road-Crossing Decisions by Autonomous Wheelchairs against Environmental Factors

Uncertainty-aware Perception Models for Off-road Autonomous Unmanned Ground Vehicles

Quality Assessment of Image Dataset for Autonomous Driving.

Image-Guided Outdoor LiDAR Perception Quality Assessment for Autonomous Driving

Exploring Credibility Scoring Metrics of Perception Systems for Autonomous Driving

A Novel Visual Perception Framework for Unmanned Aerial Vehicles: Challenges and Approaches

A Feature-based Approach for the Recognition of Image Quality Degradation in Automotive Applications

Benchmarking Image Sensors Under Adverse Weather Conditions for Autonomous Driving

Overview and Empirical Analysis of ISP Parameter Tuning for Visual Perception in Autonomous Driving

Exploring augmentation strategies in mixed reality for autonomous driving with depth cameras