Abstract:Pedestrian detection is fundamental in the realm of autonomous driving, relying on comprehensive datasets for accurate deep-learning methods. Detecting diverse pedestrian behaviors, including rare and near-accident cases, is necessary, however, there were limitations in the development of such specific datasets. Furthermore, involving humans in data collection is risky and raises privacy concerns since recording and storing data necessitates explicit consent from all participants. To tackle this challenge, we propose MetaPed, a synthetic pedestrian dataset generated from the Metaverse, where the avatars can represent a wide range of pedestrian behaviors without exposing real individuals to potential harm or privacy violations. Our Metaverse includes both programmed pedestrian behaviors and controllable avatar movements and interactions, ensuring a comprehensive and realistic representation of pedestrian actions. To validate and ensure the generalization capabilities of our synthetic dataset's performance in real-world scenarios, we perform both cross-dataset and intra-dataset evaluations on comprehensive real-world datasets, including KITTI, Citypersons, and INRIA Person. Our experiments demonstrate that the network trained on our synthetic dataset exhibits robust generalization capabilities for unseen real-world situations and our dataset's inclusion significantly enhanced the Average Precision (AP) and Average Recall (AR) of the pedestrian detection model. For cross-dataset validation, we generally observe improvements in both AP and AR with models trained on all datasets. The inclusion of our dataset also improves the intra-dataset validation of the INRIA Person dataset, where the AP and AR values increase from 0.958 to 0.988 and AR from 0.697 to 0.725, respectively. For Citypersons and KITTI datasets, the AP values increase from 0.630 to 0.639 and 0.745 to 0.752, respectively. The model training on our dataset achieved the highest AP and AR values across all test datasets and the second-highest AP observed when tested on Citypersons.

Fully Synthetic Pedestrian Anomaly Behavior Dataset Generation in Metaverse for Enhancing Autonomous Driving Object Detection

Pedestrian Detection for Autonomous Vehicles Using Virtual-to-Real Augmentation

ParGANDA: Making Synthetic Pedestrians A Reality For Object Detection

Synthetic Data Generation Framework, Dataset, and Efficient Deep Model for Pedestrian Intention Prediction

Virtual to Real adaptation of Pedestrian Detectors

Generalizable Pedestrian Detection: The Elephant In The Room

Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond

Improving the Robustness of Pedestrian Detection in Autonomous Driving with Generative Data Augmentation

Image-Level Automatic Data Augmentation for Pedestrian Detection

Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework

Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond

AutoPedestrian: An Automatic Data Augmentation and Loss Function Search Scheme for Pedestrian Detection

When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset

Deep Virtual-to-Real Distillation for Pedestrian Crossing Prediction

A Shape Transformation-based Dataset Augmentation Framework for Pedestrian Detection

Diffusion Dataset Generation: Towards Closing the Sim2Real Gap for Pedestrian Detection

Towards Rich, Portable, and Large-Scale Pedestrian Data Collection

Learning from Interaction-Enhanced Scene Graph for Pedestrian Collision Risk Assessment

PePScenes: A Novel Dataset and Baseline for Pedestrian Action Prediction in 3D

Attention-based Adversarial Appearance Learning of Augmented Pedestrians

Comparison of Pedestrian Prediction Models from Trajectory and Appearance Data for Autonomous Driving