Abstract:Transactions of the Institute of Measurement and Control, Ahead of Print. One significant challenge in improving autonomous driving algorithms is the lack of diverse real-world data. Moreover, transferring models from simulation to reality faces the reality gap problem. This study addresses this issue by developing an augmentation technique for mixed-reality environments, aimed at improving the testing and training of autonomous vehicles. Tested offline, it lays the groundwork for future online applications. The methodology focuses on creating virtual depth images using a virtual camera and applying an augmentation strategy to the KITTI data set. This creates a mixed-reality representation by combining virtual and real depth maps, leveraging depth information in the fusion process. The outcomes of this process are notably effective, achieving a balance between virtual and real-world aspects. This fusion method adeptly combines elements from both environments, maintaining the quality of the images. The novel contributions of this work include a detailed augmentation strategy that seamlessly integrates virtual objects into real depth maps, accounting for occlusions and ensuring realistic depth representations. In addition, this work demonstrates the feasibility of generating a large data set using the proposed method, significantly expanding the available data for training autonomous driving models. The use of metrics such as SSIM, peak signal-to-noise ratio (PSNR), and MAE, alongside object detection models such as Faster RCNN, provides a complete evaluation of both quantitative and qualitative aspects. The results demonstrate the quality of the augmented images, setting a foundation for potential online applications. The proposed strategy enables the generation of larger data set and facilitates safe, effective training in scenarios considered too risky or challenging to simulate accurately.

Pixel-Accurate Depth Evaluation in Realistic Driving Scenarios

A Robust Monocular Depth Estimation Framework Based on Light-Weight ERF-Pspnet for Day-Night Driving Scenes

Depth Estimation of Traffic Scenes from Image Sequence Using Deep Learning.

The ADUULM-360 Dataset -- A Multi-Modal Dataset for Depth Estimation in Adverse Weather

Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective

A synthetic digital city dataset for robustness and generalisation of depth estimation models

MonoPP: Metric-Scaled Self-Supervised Monocular Depth Estimation by Planar-Parallax Geometry in Automotive Applications

A Benchmark for Spray from Nearby Cutting Vehicles

Depth Estimation via Sparse Radar Prior and Driving Scene Semantics.

Benchmarking Image Sensors Under Adverse Weather Conditions for Autonomous Driving

Exploring augmentation strategies in mixed reality for autonomous driving with depth cameras

LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

Robust and accurate depth estimation by fusing LiDAR and Stereo

The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation

Sparse Pseudo-LiDAR Depth Assisted Monocular Depth Estimation

LED: Light Enhanced Depth Estimation at Night

DurLAR: A High-fidelity 128-channel LiDAR Dataset with Panoramic Ambient and Reflectivity Imagery for Multi-modal Autonomous Driving Applications

PLT-D3: A High-fidelity Dynamic Driving Simulation Dataset for Stereo Depth and Scene Flow

Deterministic Guided LiDAR Depth Map Completion

Real-Time Stereo Image Depth Estimation Network with Group-Wise L1 Distance for Edge Devices Towards Autonomous Driving

Depth Sensing Beyond LiDAR Range