Cytological observations on Eimeria bovis merozoites.

D. M. Hammond,J. V. Ernst,M. Goldman

DOI: https://doi.org/10.2307/3276176

1965-10-01

Journal of Parasitology

Abstract:

What problem does this paper attempt to address?

Assisting the Visually Impaired: Multitarget Warning Through Millimeter Wave Radar and RGB-depth Sensors.

Ningbo Long,Kaiwei Wang,Ruiqi Cheng,Kailun Yang,Weijian Hu,Jian Bai

DOI: https://doi.org/10.1117/1.jei.28.1.013028

IF: 0.829

2019-01-01

Journal of Electronic Imaging

Abstract:It is very difficult for visually impaired people (VIP) to perceive and avoid obstacles at a distance. To address this problem, we propose a sensor fusion system, which combines the RGB-depth (RGB-D) sensor and millimeter wave (MMW) radar sensor, to perceive the surrounding obstacles. The position and velocity information of the multiple targets are detected by the MMW radar based on the principle of frequency modulated continuous wave. The depth and position information of the obstacles are verified by the RGB-D sensor based on the MeanShift algorithm. The data fusion based on the joint probabilistic data association algorithm and Kalman filter enable the navigation assistance system to obtain more accurate state estimates compared with using only one sensor. The nonsemantic stereophonic interface is utilized to transfer the obstacle detection results to the VIP. The experiment results show that multiple objects with different ranges and angles are detected by the radar and the RGB-D sensor. The effective detection range is expanded up to 80 m compared to using only the RGB-D sensor. Moreover, the measurement results are stable under diverse illumination conditions. As a wearable system, the sensor fusion system has the characteristics of versatility, portability, and cost-effectiveness. (C) 2019 SPIE and IS&T
Fusion of millimeter wave radar and RGB-depth sensors for assisted navigation of the visually impaired

Ningbo Long,Kaiwei Wang,Ruiqi Cheng,Kailun Yang,Jian Bai

DOI: https://doi.org/10.1117/12.2324626

2018-10-05

Abstract:According to the data from the World Health Organization, 285 million people are estimated to be visually impaired worldwide, and 39 million are blind. It is very difficult for visually impaired people to perceive and avoid obstacles at a distance during their travelling. To address this problem, we propose a sensor fusion system, which combines the RGBDepth sensor and millimeter wave radar sensor, to detect the surrounding obstacles. The range and velocity of multiple obstacles are acquired by the millimeter wave radar based on the principle of frequency modulated continuous wave. The positions of the obstacles are verified by the RGB-Depth sensor based on the contour extraction and MeanShift algorithm. The data fusion algorithm based on particle filters obtains accurate state estimation by fusing RGB-Depth data with millimeter wave radar data. The experiment results show that multiple obstacles with different ranges and angles are successfully detected by the proposed system. The measurement uncertainties are reduced by the data fusion system, meanwhile the effective detectable range is expanded compared to the detection with only RGB-Depth sensor. Moreover, the measurement results are stable when the illumination varies. As a wearable prototype, the sensor fusion system has the characteristics of versatility, portability and cost-effectiveness, which is very suitable for blind navigation application.
Millimeter-Wave Radar and Vision Fusion Target Detection Algorithm Based on an Extended Network

Chunyang Qi,Chuanxue Song,Naifu Zhang,Shixin Song,Xinyu Wang,Feng Xiao

DOI: https://doi.org/10.3390/machines10080675

IF: 2.899

2022-01-01

Machines

Abstract:The need for a vehicle to perceive information about the external environmental as an independent intelligent individual has grown with the progress of intelligent driving from primary driver assistance to high-level autonomous driving. The ability of a common independent sensing unit to sense the external environment is limited by the sensor’s own characteristics and algorithm level. Hence, a common independent sensing unit fails to obtain comprehensive sensing information independently under conditions such as rain, fog, and night. Accordingly, an extended network-based fusion target detection algorithm for millimeter-wave radar and vision fusion is proposed in this work by combining the complementary perceptual performance of in-vehicle sensing elements, cost effectiveness, and maturity of independent detection technologies. Feature-level fusion is first used in this work according to the analysis of technical routes of the millimeter-wave radar and vision fusion. Training and test evaluation of the algorithm are carried out on the nuScenes dataset and test data from a homemade data acquisition platform. An extended investigation on the RetinaNet one-stage target detection algorithm based on the VGG-16+FPN backbone detection network is then conducted in this work to introduce millimeter-wave radar images as auxiliary information for visual image target detection. We use two-channel radar and three-channel visual images as inputs of the fusion network. We also propose an extended VGG-16 network applicable to millimeter-wave radar and visual fusion and an extended feature pyramid network. Test results showed that the mAP of the proposed network improves by 2.9% and the small target accuracy is enhanced by 18.73% compared with those of the reference network for pure visual image target detection. This finding verified the detection capability and algorithmic feasibility of the proposed extended fusion target detection network for visually insensitive targets.
Online Multipedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision

Fucheng Cui,Qiang Zhang,Jingxuan Wu,Yuying Song,Zhouzhen Xie,Chunyi Song,Zhiwei Xu

DOI: https://doi.org/10.1109/jsen.2023.3255924

IF: 4.3

2023-01-01

IEEE Sensors Journal

Abstract:This article focuses on the challenge of pedestrian tracking using camera and millimeter wave (MMW) radar in autonomous driving. Pedestrian tracking using a single sensor has inherent limitations due to the lack of comprehensive dimensionality of tracking information. Meanwhile, the existing multisensor-based tracking algorithms suffer from limited tracking accuracy by applying the fusion of projected positions. To enhance the tracking accuracy and robustness, a multisensor-based tracking algorithm based on fused detection of MMW radar and vision is proposed, which improves the association of detection results from multiple heterogeneous sensors by utilizing newly designed back-projection and undirected graph, and finally improves the fusion detection by simultaneously utilizing a pedestrian’s appearance, and local and global location information. Field tests are conducted to produce dataset, and the performance evaluation results based on the self-produced dataset have verified the superiority of the proposed algorithm over the conventional single-sensor-based tracking algorithm and multisensor-based tracking algorithms.
Long-Range Traversability Awareness and Low-Lying Obstacle Negotiation with RealSense for the Visually Impaired

Kailun Yang,Kaiwei Wang,Shufei Lin,Jian Bai,Luis M. Bergasa,Roberto Arroyo

DOI: https://doi.org/10.1145/3209914.3209943

2018-01-01

Abstract:The application of RGB-D sensors provides a revolutionary force in the research field of computer vision, where traversable area detection and obstacle avoidance are the fundamental topics to aid visually impaired people. However, the detection using RGB-D sensors is limited by the sparse depth map and the narrow field of view, which hampers longer traversability awareness and lower obstacle negotiation. This paper proposes an effective approach to combine a pair of wearable smart glasses and a waist-worn pathfinder to assist the visually impaired. Traversable lines are generated to feedback the visually impaired through real-time stereo sound after real-world coordinates adjustment with a RealSense R200 (RGB-D sensor) and an IMU integrated in the pair of smart glasses. Low-lying obstacles are detected by using another RGB-D sensor of RealSense, the RS410, as the waist-worn pathfinder to provide early warning. Indoor and outdoor detection results as well as a field test demonstrate the usefulness and effectiveness of our approach.
Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Qiang Zhang,Yuying Song,Zecheng Li,Fuyuan Ai,Chunyi Song,Zhiwei Xu

DOI: https://doi.org/10.1109/RadarConf2458775.2024.10548703

2024-01-01

Abstract:Multi-Object Tracking (MOT) is an important approach to perceive the surrounding environment and plays the key role in ensuring driving safety in the field of autonomous driving. However, existing MOT algorithms are facing challenges arising from the underutilization of sensor information, the neglect of sensor reliability, and the inability to effectively handle lost trajectories. This paper proposes a multi-sensor fusion algorithm based on millimeter-wave radar and visual sensor for MOT, which utilizes the average height of objects to back-project the image bounding boxes onto the bird's eye view plane. Appearance features and motion orientations are then used to expand the similarity measure between targets, adopting a multi-level data association method based on object fusion states to improve association accuracy. This also utilizes occlusion prediction and recovery methods to recover lost trajectories. For the performance evaluation, an urban environment dataset containing twenty sequences is collected through extensive experiments. The experimental results demonstrate that the proposed algorithm outperforms both the single-sensor tracking algorithm and traditional multi-sensor fusion tracking algorithm, in terms of tracking accuracy and robustness.
Low Power Millimeter Wave Radar System for the Visually Impaired

Ningbo Long,Kaiwei Wang,Ruiqi Cheng,Weijian Hu,Kailun Yang

DOI: https://doi.org/10.1049/joe.2019.0037

2019-01-01

Journal of Engineering

Abstract:In order to help visually impaired people to avoid obstacles, a K-band millimetre wave (MMW) radar system based on the principle of frequency-modulated continuous wave (FMCW) is proposed in this study. The FMCW basic principle and the signal processing algorithm are elaborated in detail. The prototype of the radar system has been designed and manufactured. The experiment results show the qualified detection accuracy in view of low deviation. The maximum deviations from the range measurement experiments for a person and a car are 0.14 and 0.47 m, respectively. As a wearable system, the radar system has the characteristics of low power and small size, which is very suitable for blind navigation application. In addition, the MMW radar has broad application prospects in the field of automotive blind spot detection and small robots.
Glass Detection and Recognition Based on the Fusion of Ultrasonic Sensor and RGB-D Sensor for the Visually Impaired

Zhiming Huang,Kaiwei Wang,Kailun Yang,Ruiqi Cheng,Jian Bai

DOI: https://doi.org/10.1117/12.2325496

2018-01-01

Abstract:With the increasing demands of visually impaired people, developing assistive technology to help them travel effectively and safely has been a research hotspot. Red, Green, Blue and Depth (RGB-D) sensor has been widely used to help visually impaired people, but the detection and recognition of glass objects is still a challenge, considering the depth information of glass cannot be obtained correctly. In order to overcome the limitation, we put forward a method to detect glass objects in natural indoor scenes in this paper, which is based on the fusion of ultrasonic sensor and RGB-D sensor on a wearable prototype. Meanwhile, the erroneous depth map of glass object computed by the RGB-D sensor could also be densely recovered. In addition, under some special circumstances, such as facing a mirror or an obstacle within the minimum detectable range of the RGB-D sensor, we use a similar processing method to regain depth information in the invalid area of the original depth map. The experimental results show that the detection range and precision of the RGB-D sensor have been significantly improved with the aid of ultrasonic sensor. The proposed method is proved to be able to detect and recognize common glass obstacles for visually impaired people in real time, which is suitable for real-world indoor navigation assistance.
Fusing LiDAR and Radar with Pillars Attention for 3D Object Detection

Hanchen Tai,Yijie Qian,Xiao Kang,Liang Liu,Yong Liu

DOI: https://doi.org/10.1109/isas61044.2024.10552581

2024-01-01

Abstract:In recent years, LiDAR has emerged as one of the primary sensors for mobile robots, enabling accurate detection of 3D objects. On the other hand, 4D millimeter-wave Radar presents several advantages which can be a complementary for LiDAR, including an extended detection range, enhanced sensitivity to moving objects, and the ability to operate seamlessly in various weather conditions, making it a highly promising technology. To leverage the strengths of both sensors, this paper proposes a novel fusion method that combines LiDAR and 4D millimeter-wave Radar for 3D object detection. The proposed approach begins with an efficient multi-modal feature extraction technique utilizing a pillar representation. This method captures comprehensive information from both LiDAR and millimeter-wave Radar data, facilitating a holistic understanding of the environment. Furthermore, a Pillar Attention Fusion (PAF) module is employed to merge the extracted features, enabling seamless integration and fusion of information from both sensors. This fusion process results in lightweight detection headers capable of accurately predicting object boxes. To evaluate the effectiveness of our proposed approach, extensive experiments were conducted on the VoD dataset. The experimental results demonstrate the superiority of our fusion method, showcasing improved performance in terms of detection accuracy and robustness across different environmental conditions. The fusion of LiDAR and 4D millimeter-wave Radar holds significant potential for enhancing the capabilities of mobile robots in real-world scenarios. The proposed method, with its efficient multi-modal feature extraction and attention-based fusion, provides a reliable and effective solution for 3D object detection.
Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Yipeng Zhu,Tao Wang,Shiqiang Zhu

DOI: https://doi.org/10.3390/rs14081837

IF: 5

2022-01-01

Remote Sensing

Abstract:Accurate and reliable tracking of multi-pedestrian is of great importance for autonomous driving, human-robot interaction and video surveillance. Since different scenarios have different best-performing sensors, sensor fusion perception plans are believed to have complementary modalities and be capable of handling situations which are challenging for single sensor. In this paper, we propose a novel track-to-track fusion strategy for multi-pedestrian tracking by using a millimeter-wave (MMW) radar and a monocular camera. Pedestrians are firstly tracked by each sensor according to the sensor characteristic. Specifically, the 3D monocular pedestrian detections are obtained by a convolutional neural network (CNN). The trajectory is formed by the tracking-by-detection approach, combined with Bayesian estimation. The measurement noise of the 3D monocular detection is modeled by a detection uncertainty value obtained from the same CNN, as an approach to estimate the pedestrian state more accurately. The MMW radar utilizes the track-before-detection method due to the sparseness of the radar features. Afterwards, the pedestrian trajectories are obtained by the proposed track-to-track fusion strategy, which can work adaptively under challenging weather conditions, low-illumination conditions and clutter scenarios. A group of tests are carried out to validate our pedestrian tracking strategy. Tracking trajectories and optimal sub-pattern assignment (OSPA) metric demonstrate the accuracy and robustness of the proposed multi-sensor multi-pedestrian tracking system.
Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

Xinyu Zhang, Mo Zhou, Peng Qiu, Yi Huang, Jun Li

DOI: https://doi.org/10.1108/IR-06-2018-0113

2019-01-01

Abstract:Purpose The purpose of this paper is the presentation and research of a novel sensor fusion-based system for obstacle detection and identification, which uses the millimeter-wave radar to detect the position and velocity of the obstacle. Afterwards, the image processing module uses the bounding box regression algorithm in deep learning to precisely locate and identify the obstacles. Design/methodology/approach Unlike the traditional algorithms that use radar and vision to detect obstacles separately, the purposed method of this paper uses radar to determine the approximate location of obstacles and then uses bounding box regression to achieve accurate positioning and recognition. First, the information of the obstacles can be acquired by the millimeter-wave radar, and the effective target is extracted by filtering the data. Then, use coordinate system conversion and camera parameter calibration to project the effective target to the image plane, and generate the region of interest (ROI). Finally, based on image processing and machine learning techniques, the vehicle targets in the ROI are detected and tracked. Findings The millimeter wave is used to determine the presence of an obstacle, and the deep learning algorithm of the image is combined to determine the shape and the class of the obstacle. The experimental results indicate that the detection rate of this method is up to 91.6 per cent, which can better implement the perception of the environment in front of the vehicle. Originality/value The originality is based on the combination of millimeter-wave sensors and deep learning. Using the bounding box regression algorithm in RCNN, the ROI detected by radar is analyzed to realize real-time obstacle detection and recognition. This method does not require processing the entire image, greatly reducing the amount of data processing and improving the efficiency of the algorithm.
Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

Kun Shi,Shibo He,Jiming Chen

DOI: https://doi.org/10.1109/apscon56343.2023.10101274

2023-01-01

Abstract:Multi-modal fusion is imperative to the implementation of reliable autonomous driving. In this paper, we make full use of both mmWave radar and camera data to reconstruct reliable depth and full-velocity information. To overcome the sparsity of radar points, we leverage full-velocity based egomotion compensation to achieve more accurate multi-sweep accumulation. Besides, we incorporate an adaptive two-stage attention module within the fusion network to exploit the synergy of camera and radar information. Furthermore, we conduct extensive experiments on the prevailing nuScenes dataset. The results show our proposed fusion system consistently outperforms the state-of-the-art methods.
Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

Zhangjing Wang,Xianhan Miao,Zhen Huang,Haoran Luo

DOI: https://doi.org/10.3390/rs13061064

IF: 5

2021-03-11

Remote Sensing

Abstract:The development of autonomous vehicles and unmanned aerial vehicles has led to a current research focus on improving the environmental perception of automation equipment. The unmanned platform detects its surroundings and then makes a decision based on environmental information. The major challenge of environmental perception is to detect and classify objects precisely; thus, it is necessary to perform fusion of different heterogeneous data to achieve complementary advantages. In this paper, a robust object detection and classification algorithm based on millimeter-wave (MMW) radar and camera fusion is proposed. The corresponding regions of interest (ROIs) are accurately calculated from the approximate position of the target detected by radar and cameras. A joint classification network is used to extract micro-Doppler features from the time-frequency spectrum and texture features from images in the ROIs. A fusion dataset between radar and camera is established using a fusion data acquisition platform and includes intersections, highways, roads, and playgrounds in schools during the day and at night. The traditional radar signal algorithm, the Faster R-CNN model and our proposed fusion network model, called RCF-Faster R-CNN, are evaluated in this dataset. The experimental results indicate that the mAP(mean Average Precision) of our network is up to 89.42% more accurate than the traditional radar signal algorithm and up to 32.76% higher than Faster R-CNN, especially in the environment of low light and strong electromagnetic clutter.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Spatial Attention Fusion for Obstacle Detection Using MmWave Radar and Vision Sensor

Shuo Chang,Yifan Zhang,Fan Zhang,Xiaotong Zhao,Sai Huang,Zhiyong Feng,Zhiqing Wei

DOI: https://doi.org/10.3390/s20040956

2020-02-11

Abstract:For autonomous driving, it is important to detect obstacles in all scales accurately for safety consideration. In this paper, we propose a new spatial attention fusion (SAF) method for obstacle detection using mmWave radar and vision sensor, where the sparsity of radar points are considered in the proposed SAF. The proposed fusion method can be embedded in the feature-extraction stage, which leverages the features of mmWave radar and vision sensor effectively. Based on the SAF, an attention weight matrix is generated to fuse the vision features, which is different from the concatenation fusion and element-wise add fusion. Moreover, the proposed SAF can be trained by an end-to-end manner incorporated with the recent deep learning object detection framework. In addition, we build a generation model, which converts radar points to radar images for neural network training. Numerical results suggest that the newly developed fusion method achieves superior performance in public benchmarking. In addition, the source code will be released in the GitHub.
Robust Target Detection of Intelligent Integrated Optical Camera and mmWave Radar System

Chen Zhu,Zhouxiang Zhao,Zejing Shan,Lijie Yang,Sijie Ji,Zhaohui Yang,Zhaoyang Zhang

DOI: https://doi.org/10.1016/j.dsp.2023.104336

2023-12-12

Abstract:Target detection is pivotal for modern urban computing applications. While image-based techniques are widely adopted, they falter under challenging environmental conditions such as adverse weather, poor lighting, and occlusion. To improve the target detection performance under complex real-world scenarios, this paper proposes an intelligent integrated optical camera and millimeter-wave (mmWave) radar system. Utilizing both physical knowledge and data-driven methods, a long-term robust radar-camera fusion algorithm is proposed to solve the heterogeneous data fusion problem for detection improvement. For the occlusion scenarios, the proposed algorithm can effectively detect occluded targets with the help of memory through performing long-term detection. For dark scenarios with low-light conditions, the proposed algorithm can effectively mark the target in the dark picture as well as provide rough stickman imaging. The above two innovative functions of the hybrid optical camera and mmWave radar system are tested in real-world scenarios. The results demonstrate the robustness and significant enhancement in the target detection performance of our integrated system.

Image and Video Processing
Bridging the View Disparity Between Radar and Camera Features for Multi-Modal Fusion 3D Object Detection

Taohua Zhou,Junjie Chen,Yining Shi,Kun Jiang,Mengmeng Yang,Diange Yang

DOI: https://doi.org/10.1109/tiv.2023.3240287

IF: 8.2

2023-02-01

IEEE Transactions on Intelligent Vehicles

Abstract:Environmental perception with the multi-modal fusion is crucial in autonomous driving to increase accuracy, completeness, and robustness. This paper focuses on utilizing millimeter-wave (MMW) radar and camera sensor fusion for 3D object detection. A novel method that realizes the feature-level fusion under the bird's-eye view (BEV) for a better feature representation is proposed. Firstly, radar points are augmented with temporal accumulation and sent to a spatial-temporal encoder for radar feature extraction. Meanwhile, multi-scale image 2D features which adapt to various spatial scales are obtained by image backbone and neck model. Then, image features are transformed to BEV with the designed view transformer. In addition, this work fuses the multi-modal features with a two-stage fusion model called point-fusion and ROI-fusion, respectively. Finally, a detection head regresses objects category and 3D locations. Experimental results demonstrate that the proposed method realizes the state-of-the-art (SOTA) performance under the most crucial detection metricsmean average precision (mAP) and nuScenes detection score (NDS) on the challenging nuScenes dataset.
Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera

Jianyu Duan

DOI: https://doi.org/10.3390/s23136044

IF: 3.9

2023-07-01

Sensors

Abstract:This study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to predict target motion more accurately and an improved joint probability data association method to match measurement data with targets. An adaptive root-mean-square cubature Kalman filter is used to estimate the statistical characteristics of noise under complex traffic scenarios with varying process and measurement noise. Experiments conducted on a real vehicle platform demonstrate that the proposed framework improves reliability and robustness in challenging environments. It overcomes the challenges of insufficient data fusion utilization, frequent leakage, and misjudgment of dangerous obstructions around vehicles, and inaccurate prediction of collision risks. The proposed framework has the potential to advance the state of the art in target tracking and perception for intelligent vehicles.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera

Pengfei Lv,Bingqing Wang,Feng Cheng,Jinlin Xue

DOI: https://doi.org/10.3390/s23010230

IF: 3.9

2022-12-27

Sensors

Abstract:In order to remedy the defects of single sensor in robustness, accuracy, and redundancy of target detection, this paper proposed a method for detecting obstacles in farmland based on the information fusion of a millimeter wave (mmWave) radar and a camera. Combining the advantages of the mmWave radar in range and speed measurement and the camera in type identification and lateral localization, a decision-level fusion algorithm was designed for the mmWave radar and camera information, and the global nearest neighbor method was used for data association. Then, the effective target sequences of the mmWave radar and the camera with successful data association were weighted to output, and the output included more accurate target orientation, longitudinal speed, and category. For the unassociated sequences, they were tracked as new targets by using the extended Kalman filter algorithm and were processed and output during the effective life cycle. Lastly, an experimental platform based on a tractor was built to verify the effectiveness of the proposed association detection method. The obstacle detection test was conducted under the ROS environment after solving the external parameters of the mmWave radar and the internal and external parameters of the camera. The test results show that the correct detection rate of obstacles reaches 86.18%, which is higher than that of a single camera with 62.47%. Furthermore, through the contrast experiment of the sensor fusion algorithms, the detection accuracy of the decision level fusion algorithm was 95.19%, which was higher than 4.38% and 6.63% compared with feature level and data level fusion, respectively.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Nighttime Pedestrian Detection Based on a Fusion of Visual Information and Millimeter-Wave Radar

Wei Zhao,Tingting Wang,Ao Tan,Congcong Ren

DOI: https://doi.org/10.1109/access.2023.3291398

IF: 3.9

2023-07-15

IEEE Access

Abstract:A target detection algorithm based on the fusion of vision sensors and millimeter wave radar data can effectively improve the safety of self-driving vehicles. However, a single sensor cannot obtain comprehensive category status information on the target in the nighttime environment. To improve pedestrian detection in nighttime traffic scenarios, this paper proposes a nighttime pedestrian detection method based on the fusion of infrared vision information and millimeter wave (MMW) radar data. The lateral localization and category features of the target are obtained using the improved YOLOv5 deep learning algorithm, the distance and velocity information of the target is obtained by preprocessing the MMW radar acquisition data, the pedestrian target is tracked by using the extended Kalman filter for MMW radar detection, the projection of the valid radar target point on the Infrared Radiatio (IR) image is completed according to the spatiotemporal fusion, and then the correlation gate method is used to correlate the data, and the visual information is inherited to the radar points to get the target multimodal information, and the success associated valid target sequences are weighted to obtain the accurate target position. Finally, a decision-level fusion algorithm framework is proposed to complete the output of pedestrian multimodal information in nighttime traffic scenes. Theoretical analysis and experimental results show that the accuracy and robustness of this method for nighttime pedestrian recognition are better than those of a single sensor.

computer science, information systems,telecommunications,engineering, electrical & electronic

Cytological observations on Eimeria bovis merozoites.

Assisting the Visually Impaired: Multitarget Warning Through Millimeter Wave Radar and RGB-depth Sensors.

Fusion of millimeter wave radar and RGB-depth sensors for assisted navigation of the visually impaired

Millimeter-Wave Radar and Vision Fusion Target Detection Algorithm Based on an Extended Network

Online Multipedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision

Long-Range Traversability Awareness and Low-Lying Obstacle Negotiation with RealSense for the Visually Impaired

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Low Power Millimeter Wave Radar System for the Visually Impaired

Glass Detection and Recognition Based on the Fusion of Ultrasonic Sensor and RGB-D Sensor for the Visually Impaired

Fusing LiDAR and Radar with Pillars Attention for 3D Object Detection

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

Spatial Attention Fusion for Obstacle Detection Using MmWave Radar and Vision Sensor

Robust Target Detection of Intelligent Integrated Optical Camera and mmWave Radar System

Bridging the View Disparity Between Radar and Camera Features for Multi-Modal Fusion 3D Object Detection

Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera

Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera

Nighttime Pedestrian Detection Based on a Fusion of Visual Information and Millimeter-Wave Radar