Abstract:Advanced driver assistance systems (ADASs) and autonomous vehicles rely on different types of sensors, such as camera, radar, ultrasonic, and LiDAR, to sense the surrounding environment. Compared with the other types of sensors, millimeter-wave automotive radar has advantages in terms of cost and reliability under bad weather conditions (e.g., snow, rain, and fog) and does not suffer from light condition variations (e.g., darkness). Typical radar devices used in today's commercial vehicles with ADAS features produce sparse point clouds in low angular resolution with a limited number of antennas. In this article, we present a machine-learning-aided signal processing chain to suppress the radar imaging blur effect introduced by the phase migration in time-division multiplexing multiple-input multiple-output radar, to generate low-level high-resolution radar bird's-eye view (BEV) spectra with rich object's features. Compared with radar point clouds, there is no information loss in radar BEV spectra. We then propose a temporal-fusion distance-tolerant single-stage object detection network, termed as TDRadarNet, and an enhanced version, TDRadarNet+, to robustly detect vehicles in both long and short ranges on radar BEVs. We introduce a first-of-its-kind multimodel dataset, containing 14 800 frames of high-resolution low-level radar BEV spectra with synchronized stereo camera RGB images and 3-D LiDAR point clouds. Our dataset achieves 0.39-m range resolution and 1.2 degrees degree azimuth angular resolution with 100-m maximum detectable range. Moreover, we create a subdataset, the Doppler Unfolding dataset, containing 244 140 beam vectors extracted from the 3-D radar data cube. With extensive testing and evaluation, we demonstrate that our Doppler unfolding network achieves 93.46% Doppler unfolding accuracy. Compared to YOLOv7, a state-of-the-art image-based object detection network, TDRadarNet, achieves a 70.3% average precision (AP) for vehicle detection, demonstrating a 21.0% improvement; TDRadarNet+ achieves a 73.9% AP, showing a 24.6% improvement in performance.

mmWave-YOLO: A mmWave Imaging Radar-Based Real-Time Multiclass Object Recognition System for ADAS Applications

Point Cloud Features-Based Kernel SVM for Human-Vehicle Classification in Millimeter Wave Radar.

MS-YOLO: Object Detection Based on YOLOv5 Optimized Fusion Millimeter-wave Radar and Machine Vision

Target Recognition of Millimeter-wave Radar based on YOLOX

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

A Combined mmWave Tracking and Classification Framework Using a Camera for Labeling and Supervised Learning

Deep Learning Derived Object Detection and Tracking Technology Based on Sensor Fusion of Millimeter-Wave Radar/Video and Its Application on Embedded Systems

A Systematic Study on Object Recognition Using Millimeter-wave Radar

mmDetect: YOLO-based Processing of mm-Wave Radar Data for Detecting Moving People

Yolov4-based hybrid feature enhancement network with robust object detection under adverse weather conditions

Human Detection and Action Classification Based on Millimeter Wave Radar Point Cloud Imaging Technology

Deep-Neural-Network-Enabled Vehicle Detection Using High-Resolution Automotive Radar Imaging

MVFAN: Multi-View Feature Assisted Network for 4D Radar Object Detection

Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

Concealed Object Detection and Recognition System Based on Millimeter Wave FMCW Radar

RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition

Target Detection and Recognition in Synthetic Aperture Radar Images Using YOLO Deep Learning Methods

Automatic Target Recognition of Millimeter-Wave Radar Based on Deep Learning

A New Automotive Radar 4D Point Clouds Detector by Using Deep Learning

Beyond Point Clouds: A Knowledge-Aided High Resolution Imaging Radar Deep Detector for Autonomous Driving