Sensor Data Fusion Based on Deep Learning for Computer Vision Applications and Medical Applications

Rizwan Ali Naqvi,Muhammad Arsalan,Talha Qaiser,Tariq Mahmood Khan,Imran Razzak

DOI: https://doi.org/10.3390/s22208058

2022-10-21

Abstract:Sensor fusion is the process of merging data from many sources, such as radar, lidar and camera sensors, to provide less uncertain information compared to the information collected from single source [...].

What problem does this paper attempt to address?

Sensor Data Fusion Analysis for Broad Applications

Natividad Duro

DOI: https://doi.org/10.3390/s24123725

IF: 3.9

2024-06-08

Sensors

Abstract:Sensor data fusion analysis plays a pivotal role in a variety of fields by integrating data from multiple sensors to produce more accurate, reliable, and comprehensive information than that achieved by individual sensors alone [...]

engineering, electrical & electronic,instruments & instrumentation,chemistry, analytical
Object Detection Using Multi-Sensor Fusion Based on Deep Learning

Taohua Zhou,Kun Jiang,Zhongyang Xiao,Chunlei Yu,Diange Yang

DOI: https://doi.org/10.1061/9780784482292.496

2019-01-01

Abstract:Autonomous driving is an important part of intelligent traffic. It requires accurate and effective environmental perception. Fusion of multi-sensor perception information is an effective way to achieve environmental perception. This paper studies information fusion of millimeter-wave radar and camera, mainly using deep learning method to achieve traffic object detection. The detection result is directly used to make driving decisions. We constructed a perception data collection system and used spatial-temporal synchronization to associate multi-sensor data. Then we design a type of deep fusion algorithm. Based on YOLOv2 (you only look once V2) algorithm, we modified the neural network to implement deep fusion. We applied a method to input multi-sensor data into CNN (convolutional neural networks) using a KITTI dataset. The model can provide useful object information such as category and location. By comparing our results with the result of classical post-fusion algorithm, we found that deep fusion is more efficient.
Deep Learning Algorithm for Optimized Sensor Data Fusion in Fault Diagnosis and Tolerance

M. Elhoseny,Deepak Dasaratha Rao,Bala Dhandayuthapani Veerasamy,Noha Alduaiji,J. Shreyas,Piyush Kumar Shukla

DOI: https://doi.org/10.1007/s44196-024-00692-5

IF: 2.259

2024-12-04

International Journal of Computational Intelligence Systems

Abstract:Environmental perception is one of the key technologies to realize autonomous vehicles. The fault diagnosis process involves identifying the fault that occurred or the cause of the out-of-control condition. Here, the major objective is to locate problems in detection by analysing previous data or sequential patterns of data that cause failure. This study evaluates the use of deep learning for improved sensor data fusion in fault identification and tolerance using the KITTI dataset. The input video from the dataset has been transformed to frames through median filtering. Next, feature extraction is applied to a preprocessed image, resulting in the fusion of sensor data. Data fusion is then carried out utilizing an enhanced RPN (region proposal network). The enhanced RPN also has a loss function (object detection loss, bounding box loss and target classification loss), an estimate of ROI and feature extraction network (FEN). Through the use of the COOT connected blue monkey optimization (CCBMO) model, the weight of the optimally enhanced RPN is established. Next, using global non-maximum suppression with both global and local confidence, fault identification and tolerance are carried out. From the analysis, it clearly shows that proposed method accomplished better results in terms of accuracy, precision and specificity of 97.78%, 93.76% and 93.43%, respectively, when compared with various conventional models with respect to diverse performance measures.

computer science, artificial intelligence, interdisciplinary applications
Deepfusion: A Deep Learning Framework For The Fusion Of Heterogeneous Sensory Data

Hongfei Xue,Wenjun Jiang,Chenglin Miao,Ye Yuan,Fenglong Ma,Xin Ma,Yijiang Wang,Shuochao Yao,Wenyao Xu,Aidong Zhang,Lu Su

DOI: https://doi.org/10.1145/3323679.3326513

2019-01-01

Abstract:In recent years, significant research efforts have been spent towards building intelligent and user-friendly IoT systems to enable a new generation of applications capable of performing complex sensing and recognition tasks. In many of such applications, there are usually multiple different sensors monitoring the same object. Each of these sensors can be regarded as an information source and provides us a unique "view" of the observed object. Intuitively, if we can combine the complementary information carried by multiple sensors, we will be able to improve the sensing performance. Towards this end, we propose DeepFusion, a unified multi-sensor deep learning framework, to learn informative representations of heterogeneous sensory data. DeepFusion can combine different sensors' information weighted by the quality of their data and incorporate cross-sensor correlations, and thus can benefit a wide spectrum of IoT applications. To evaluate the proposed DeepFusion model, we set up two real-world human activity recognition testbeds using commercialized wearable and wireless sensing devices. Experiment results show that DeepFusion can outperform the state-of-the-art human activity recognition methods.
Deep Sensor Fusion for Real-Time Odometry Estimation

Michelle Valente,Cyril Joly,Arnaud de La Fortelle

DOI: https://doi.org/10.48550/arXiv.1908.00524

2019-07-31

Abstract:Cameras and 2D laser scanners, in combination, are able to provide low-cost, light-weight and accurate solutions, which make their fusion well-suited for many robot navigation tasks. However, correct data fusion depends on precise calibration of the rigid body transform between the sensors. In this paper we present the first framework that makes use of Convolutional Neural Networks (CNNs) for odometry estimation fusing 2D laser scanners and mono-cameras. The use of CNNs provides the tools to not only extract the features from the two sensors, but also to fuse and match them without needing a calibration between the sensors. We transform the odometry estimation into an ordinal classification problem in order to find accurate rotation and translation values between consecutive frames. Results on a real road dataset show that the fusion network runs in real-time and is able to improve the odometry estimation of a single sensor alone by learning how to fuse two different types of data information.

Robotics,Computer Vision and Pattern Recognition
A Review on Multisensor Data Fusion for Wearable Health Monitoring

Arlene John,Barry Cardiff,Deepu John

2024-12-08

Abstract:The growing demand for accurate, continuous, and non-invasive health monitoring has propelled multi-sensor data fusion to the forefront of healthcare technology. This review aims to provide an overview of the development of fusion frameworks in the literature and common terminology used in fusion literature. The review introduces the fusion classification standards and methods that are most relevant from an algorithm development perspective. Applications of the reviewed fusion frameworks in fields such as defense, autonomous driving, robotics, and image fusion are also discussed to provide contextual information on the various fusion methodologies that have been developed in this field. This review provides a comprehensive analysis of multi-sensor data fusion methods applied to health monitoring systems, focusing on key algorithms, applications, challenges, and future directions. We examine commonly used fusion techniques, including Kalman filters, Bayesian networks, and machine learning models. By integrating data from various sources, these fusion approaches enhance the reliability, accuracy, and resilience of health monitoring systems. However, challenges such as data quality and differences in acquisition systems exist, calling for intelligent fusion algorithms in recent years. The review finally converges on applications of fusion algorithms in biomedical inference tasks like heartbeat detection, respiration rate estimation, sleep apnea detection, arrhythmia detection, and atrial fibrillation detection.

Signal Processing
Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

Numan Senel,Klaus Kefferpütz,Kristina Doycheva,Gordon Elger

DOI: https://doi.org/10.3390/pr11020501

IF: 3.5

2023-02-07

Processes

Abstract:Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.

engineering, chemical
A Novel Multi-Sensor Hybrid Fusion Framework

Haoran Du,Qi Wang,Xunan Zhang,Wenjun Qian,Jixin Wang

DOI: https://doi.org/10.1088/1361-6501/ad42c4

IF: 2.398

2024-04-26

Measurement Science and Technology

Abstract:Multi-sensor data fusion has emerged as a powerful approach to enhance the accuracy and robustness of diagnostic systems. However, effectively integrating multiple sensor data remains a challenge. To address this issue, this paper proposes a novel multi-sensor fusion framework. Firstly, a vibration signal weighted fusion rule based on Kullback-Leibler (K-L) Divergence-Permutation Entropy (PE) is introduced, which adaptively determines the weighting coefficients by considering the positional differences of different sensors. Secondly, a lightweight multi-scale convolutional neural network is designed for feature extraction and fusion of multi-sensor data. An ensemble classifier is employed for fault classification, and an improved hard voting strategy is proposed to achieve more reliable decision fusion. Finally, the superiority of the proposed method is validated using modular state detection data from the Kaggle database.

engineering, multidisciplinary,instruments & instrumentation
Deep Learning for Pixel-Level Image Fusion: Recent Advances and Future Prospects

Yu Liu,Xun Chen,Zengfu Wang,Z. Jane Wang,Rabab K. Ward,Xuesong Wang

DOI: https://doi.org/10.1016/j.inffus.2017.10.007

IF: 18.6

2017-01-01

Information Fusion

Abstract:By integrating the information contained in multiple images of the same scene into one composite image, pixel-level image fusion is recognized as having high significance in a variety of fields including medical imaging, digital photography, remote sensing, video surveillance, etc. In recent years, deep learning (DL) has achieved great success in a number of computer vision and image processing problems. The application of DL techniques in the field of pixel-level image fusion has also emerged as an active topic in the last three years. This survey paper presents a systematic review of the DL-based pixel-level image fusion literature. Specifically, we first summarize the main difficulties that exist in conventional image fusion research and discuss the advantages that DL can offer to address each of these problems. Then, the recent achievements in DL-based image fusion are reviewed in detail. More than a dozen recently proposed image fusion methods based on DL techniques including convolutional neural networks (CNNs), convolutional sparse representation (CSR) and stacked autoencoders (SAEs) are introduced. At last, by summarizing the existing DL-based image fusion methods into several generic frameworks and presenting a potential DL-based framework for developing objective evaluation metrics, we put forward some prospects for the future study on this topic. The key issues and challenges that exist in each framework are discussed.
Learnable fusion mechanisms for multimodal object detection in autonomous vehicles

Yahya Massoud,Robert Laganiere

DOI: https://doi.org/10.1049/cvi2.12259

IF: 1.484

2024-03-17

IET Computer Vision

Abstract:The authors propose a deep learning‐based sensor fusion framework that uses both camera and LiDAR sensors in a multi‐modal and multi‐view setting. In order to leverage both data streams, two fusion mechanisms are incorporated: element‐wise multiplication and multi‐modal factorised bilinear pooling. The authors provide a detailed study of important design choices that contribute to the performance of deep learning‐based sensor fusion frameworks such as data augmentation, multi‐task learning, and the design of convolutional architecture. Perception systems in autonomous vehicles need to accurately detect and classify objects within their surrounding environments. Numerous types of sensors are deployed on these vehicles, and the combination of such multimodal data streams can significantly boost performance. The authors introduce a novel sensor fusion framework using deep convolutional neural networks. The framework employs both camera and LiDAR sensors in a multimodal, multiview configuration. The authors leverage both data types by introducing two new innovative fusion mechanisms: element‐wise multiplication and multimodal factorised bilinear pooling. The methods improve the bird's eye view moderate average precision score by +4.97% and +8.35% on the KITTI dataset when compared to traditional fusion operators like element‐wise addition and feature map concatenation. An in‐depth analysis of key design choices impacting performance, such as data augmentation, multi‐task learning, and convolutional architecture design is offered. The study aims to pave the way for the development of more robust multimodal machine vision systems. The authors conclude the paper with qualitative results, discussing both successful and problematic cases, along with potential ways to mitigate the latter.

computer science, artificial intelligence,engineering, electrical & electronic
Multi-Sensor Fusion in Automated Driving: A Survey

Zhangjing Wang,Yu Wu,Qingqing Niu

DOI: https://doi.org/10.1109/access.2019.2962554

IF: 3.9

2020-01-01

IEEE Access

Abstract:With the significant development of practicability in deep learning and the ultra-high-speed information transmission rate of 5G communication technology will overcome the barrier of data transmission on the Internet of Vehicles, automated driving is becoming a pivotal technology affecting the future industry. Sensors are the key to the perception of the outside world in the automated driving system and whose cooperation performance directly determines the safety of automated driving vehicles. In this survey, we mainly discuss the different strategies of multi-sensor fusion in automated driving in recent years. The performance of conventional sensors and the necessity of multi-sensor fusion are analyzed, including radar, LiDAR, camera, ultrasonic, GPS, IMU, and V2X. According to the differences in the latest studies, we divide the fusion strategies into four categories and point out some shortcomings. Sensor fusion is mainly applied for multi-target tracking and environment reconstruction. We discuss the method of establishing a motion model and data association in multi-target tracking. At the end of the paper, we analyzed the deficiencies in the current studies and put forward some suggestions for further improvement in the future. Through this investigation, we hope to analyze the current situation of multi-sensor fusion in the automated driving process and provide more efficient and reliable fusion strategies.

computer science, information systems,telecommunications,engineering, electrical & electronic
DATA FUSION OF ULTRASONIC SENSORS FOR DISTANCE MEASUREMENT

Mahabbat Khudaverdieva Mahabbat Khudaverdieva

DOI: https://doi.org/10.36962/piretc29082023-43

2023-11-06

PIRETC-Proceeding of The International Research Education & Training Centre

Abstract:Distance measurement plays a fundamental role in contemporary technology, exerting a significant influence on applications across a myriad of sectors, ranging from autonomous navigation and industrial automation to robotics. Ultrasonic sensors, celebrated for their simplicity, cost-efficiency, and adaptability, have grown into essential instruments for gauging distances. Nevertheless, individual ultrasonic sensors are not exempt from constraints, which encompass inaccuracies stemming from environmental conditions, sensor noise, and measurement errors. This article delves into the advanced practice of data fusion, a method that entails amalgamating data from multiple ultrasonic sensors, bolstered by intricate algorithms like artificial intelligence (AI) and statistical techniques. The fusion of ultrasonic sensor data surpasses mere amalgamation; it harnesses the potency of AI to refine data integration. Consequently, this doesn't only enhance measurement precision but also equips systems to function dependably in dynamic and demanding environments. The article explores the importance of ultrasonic sensors, the necessity for data fusion, procedures for data preprocessing, the extraction of relevant features, the application of fusion algorithms, and the handling of uncertainties. It investigates the utilization of ultrasonic sensor data fusion in diverse domains such as autonomous vehicles, robotics, industrial automation, healthcare, and environmental monitoring. Moreover, it confronts issues associated with data quality, environmental variations, privacy concerns, and ethical deliberations. The article concludes by shedding light on the encouraging avenues that lie ahead in the field, which include the development of advanced algorithms, the seamless integration of different sensor types, real-time data processing, and a strong emphasis on safety-critical applications. Data fusion from ultrasonic sensors signifies an evolving and transformative technology with profound consequences for various sectors. The responsible advancement and deployment of this technology are of paramount importance to ensure the realization of its full potential, while maintaining the principles of safety, ethics, and reliability in a continuously evolving technological landscape. Keywords: Complex measurement, Multi-sensor distance measurement, Ultrasonic sensors, Data fusion, Measurement accuracy.
Learning Selective Sensor Fusion for States Estimation

Changhao Chen,Stefano Rosa,Chris Xiaoxuan Lu,Bing Wang,Niki Trigoni,Andrew Markham

DOI: https://doi.org/10.48550/arXiv.1912.13077

2022-05-18

Abstract:Autonomous vehicles and mobile robotic systems are typically equipped with multiple sensors to provide redundancy. By integrating the observations from different sensors, these mobile agents are able to perceive the environment and estimate system states, e.g. locations and orientations. Although deep learning approaches for multimodal odometry estimation and localization have gained traction, they rarely focus on the issue of robust sensor fusion - a necessary consideration to deal with noisy or incomplete sensor observations in the real world. Moreover, current deep odometry models suffer from a lack of interpretability. To this extent, we propose SelectFusion, an end-to-end selective sensor fusion module which can be applied to useful pairs of sensor modalities such as monocular images and inertial measurements, depth images and LIDAR point clouds. Our model is a uniform framework that is not restricted to specific modality or task. During prediction, the network is able to assess the reliability of the latent features from different sensor modalities and estimate trajectory both at scale and global pose. In particular, we propose two fusion modules - a deterministic soft fusion and a stochastic hard fusion, and offer a comprehensive study of the new strategies compared to trivial direct fusion. We extensively evaluate all fusion strategies in both public datasets and on progressively degraded datasets that present synthetic occlusions, noisy and missing data and time misalignment between sensors, and we investigate the effectiveness of the different fusion strategies in attending the most reliable features, which in itself, provides insights into the operation of the various models.

Computer Vision and Pattern Recognition,Machine Learning,Robotics
Information Fusion of Multi-Sensor Images

Y.J. ZHANG

DOI: https://doi.org/10.4018/978-1-60566-026-4.ch307

2009-01-01

Abstract:The human perception to the outside world is the results of action among brain and many organs. For example, the intelligent robots that people currently investigate can have many sensors for sense of vision, sense of hearing, sense of taste, sense of smell, sense of touch, sense of pain, sense of heat, sense of force, sense of slide, sense of approach (Luo, 2002). All these sensors provide different profile information of scene in same environment. To use suitable techniques for assorting with various sensors and combining their obtained information, the theories and methods of multi-sensor fusion are required. Multi-sensor information fusion is a basic ability of human beings. Single sensor can only provide incomplete, un-accurate, vague, uncertainty information. Sometimes, information obtained by different sensors can even be contradictory. Human beings have the ability to combine the information obtained by different organs and then make estimation and decision for environment and events. Using computer to perform multi-sensor information fusion can be considered as a simulation of the function of human brain for treating complex problems. Multi-sensor information fusion consists of operating on the information data come from various sensors and obtaining more comprehensive, accurate, and robust results than that obtained from single sensor. Fusion can be defined as the process of combined treating of data acquired from multiple sensors, as well as assorting, optimizing and conforming of these data to increase the ability of extracting information and improving the decision capability. Fusion can extend the coverage for space and time information, reducing the fuzziness, increasing the reliability of making decision, and the robustness of systems. Image fusion is a particular type of multi-sensor fusion, which takes images as operating objects. In a more general sense of image engineering (Zhang, 2006), the combination of multi-resolution images also can be counted as a fusion process. In this article, however, the emphasis is put on the information fusion of multi-sensor images.
Multimodal Medical Image Fusion: The Perspective of Deep Learning

Mingyang Wei,Mengbo Xi,Yabei Li,Minjun Liang,Ge Wang

DOI: https://doi.org/10.54097/ajst.v5i3.8013

2023-05-05

Academic Journal of Science and Technology

Abstract:Multimodal medical image fusion involves the integration of medical images originating from distinct modalities and captured by various sensors, with the aim to enhance image quality, minimize redundant information, and preserve specific features, ultimately leading to increased efficiency and accuracy in clinical diagnoses. In recent years, the emergence of deep learning techniques has propelled significant advancements in image fusion, addressing the limitations of conventional methods that necessitate manual design of activity level measurement and fusion rules. This paper initially presents a systematic description of the multimodal medical image fusion problem, delineating the interrelationships between different fusion modalities while summarizing their characteristics and functions. Subsequently, it reviews the theories and enhancement approaches associated with deep learning in the medical image fusion domain, striving for a comprehensive overview of the state-of-the-art developments in this field from a deep learning perspective. These developments encompass multimodal feature extraction methods based on convolutional techniques, adversarial learning-based methods, convolutional sparse representation and stacked autoencoder-based signal processing methods, and unified models. Lastly, the paper summarizes the enhancement techniques for multimodal medical image fusion methods, highlighting the pressing issues and challenges encountered by deep learning approaches in this domain.
Enabling Efficient Deep Convolutional Neural Network-based Sensor Fusion for Autonomous Driving

Xiaoming Zeng,Zhendong Wang,Yang Hu

DOI: https://doi.org/10.48550/arXiv.2202.11231

2022-02-23

Abstract:Autonomous driving demands accurate perception and safe decision-making. To achieve this, automated vehicles are now equipped with multiple sensors (e.g., camera, Lidar, etc.), enabling them to exploit complementary environmental context by fusing data from different sensing modalities. With the success of Deep Convolutional Neural Network(DCNN), the fusion between DCNNs has been proved as a promising strategy to achieve satisfactory perception accuracy. However, mainstream existing DCNN fusion schemes conduct fusion by directly element-wisely adding feature maps extracted from different modalities together at various stages, failing to consider whether the features being fused are matched or not. Therefore, we first propose a feature disparity metric to quantitatively measure the degree of feature disparity between the feature maps being fused. We then propose Fusion-filter as a feature-matching techniques to tackle the feature-mismatching issue. We also propose a Layer-sharing technique in the deep layer that can achieve better accuracy with less computational overhead. Together with the help of the feature disparity to be an additional loss, our proposed technologies enable DCNN to learn corresponding feature maps with similar characteristics and complementary visual context from different modalities to achieve better accuracy. Experimental results demonstrate that our proposed fusion technique can achieve better accuracy on KITTI dataset with less computational resources demand.

Computer Vision and Pattern Recognition
Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles

Babak Shahian Jahromi,Theja Tulabandhula,Sabri Cetin

DOI: https://doi.org/10.3390/s19204357

IF: 3.9

2019-10-09

Sensors

Abstract:There are many sensor fusion frameworks proposed in the literature using different sensors and fusion methods combinations and configurations. More focus has been on improving the accuracy performance; however, the implementation feasibility of these frameworks in an autonomous vehicle is less explored. Some fusion architectures can perform very well in lab conditions using powerful computational resources; however, in real-world applications, they cannot be implemented in an embedded edge computer due to their high cost and computational need. We propose a new hybrid multi-sensor fusion pipeline configuration that performs environment perception for autonomous vehicles such as road segmentation, obstacle detection, and tracking. This fusion framework uses a proposed encoder-decoder based Fully Convolutional Neural Network (FCNx) and a traditional Extended Kalman Filter (EKF) nonlinear state estimator method. It also uses a configuration of camera, LiDAR, and radar sensors that are best suited for each fusion method. The goal of this hybrid framework is to provide a cost-effective, lightweight, modular, and robust (in case of a sensor failure) fusion system solution. It uses FCNx algorithm that improve road detection accuracy compared to benchmark models while maintaining real-time efficiency that can be used in an autonomous vehicle embedded computer. Tested on over 3K road scenes, our fusion algorithm shows better performance in various environment scenarios compared to baseline benchmark networks. Moreover, the algorithm is implemented in a vehicle and tested using actual sensor data collected from a vehicle, performing real-time environment perception.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Multimodal medical image fusion and classification using deep learning techniques

D. Veeraiah,S. Sai Kumar,Rajendra Kumar Ganiya,Katta Subba Rao,J. Nageswara Rao,Ramaswamy Manjith,A. Rajaram

DOI: https://doi.org/10.3233/jifs-240018

2024-04-20

Journal of Intelligent & Fuzzy Systems

Abstract:Medical image fusion plays a crucial role in accurate medical diagnostics by combining images from various modalities. To address this need, we propose an AI model for efficient medical image fusion using multiple modalities. Our approach utilizes a Siamese convolutional neural network to construct a weight map based on pixel movement information extracted from multimodality medical images. We leverage medical picture pyramids to incorporate multiscale techniques, enhancing reliability beyond human visual intuition. Additionally, we dynamically adjust the fusion mode based on local comparisons of deconstructed coefficients. Evaluation metrics including F1-score, recall, accuracy, and precision are computed to assess performance, yielding impressive results: an F1-score of 0.8551 and a mutual information (MI) value of 2.8059. Experimental results demonstrate the superiority of our method, achieving a remarkable 99.61% accuracy in targeted experiments. Moreover, the Structural Similarity Index (SSIM) of our approach is 0.8551. Compared to state-of-the-art approaches, our model excels in medical picture classification, providing accurate diagnosis through high-quality fused images. This research advances medical image fusion techniques, offering a robust solution for precise medical diagnostics across various modalities.
Autonomous Multi-Sensor Fusion Techniques for Environmental Perception in Self-Driving Vehicles

Gowtham Raj R,Prateek Chaturvedi,Shruti Patil,Ali Abduhussien Hameed,H. Thethi,Ippa Sumalatha

DOI: https://doi.org/10.1109/IC3SE62002.2024.10593125

2024-05-09

Abstract:Assuming strong environmental awareness, the arrival of autonomous vehicles signals a revolutionary age in transportation. In order for autonomous vehicles to have trustworthy perception, this study explores the complex field of autonomous multi-sensor fusion methods. It highlights how different types of sensors, such as LiDAR, radar, and cameras, may work together to provide more complete and accurate environmental data. Examining the relative merits and practicality of early, late, and deep fusion, the article methodically investigates the several sensor fusion systems and how they fare in complicated and dynamic driving situations. The adaptive fusion technique, which improves detection and classification accuracy and optimizes computational resources by dynamically adjusting to ambient variables, is a novel component of this study. Intelligent data integration and interpretation from sensors is the goal of this strategy, which makes use of machine learning techniques like recurrent neural networks and convolutional neural networks. Results show that the suggested method outperforms the state-of-the-art when compared to well-known benchmarks in terms of accuracy, reliability, and computing efficiency. In addition, this study emphasizes the important problems and potential solutions related to multi-sensor fusion, including handling data that is not updated in real-time, overcoming bad weather, and guaranteeing the ability to analyze data in real-time. To realize self-driving vehicles' full potential and create smarter, more efficient transportation networks, the results show how crucial new fusion techniques are.

Environmental Science,Engineering,Computer Science

Sensor Data Fusion Based on Deep Learning for Computer Vision Applications and Medical Applications

Sensor Data Fusion Analysis for Broad Applications

Object Detection Using Multi-Sensor Fusion Based on Deep Learning

Deep Learning Algorithm for Optimized Sensor Data Fusion in Fault Diagnosis and Tolerance

Deepfusion: A Deep Learning Framework For The Fusion Of Heterogeneous Sensory Data

Deep Sensor Fusion for Real-Time Odometry Estimation

A Review on Multisensor Data Fusion for Wearable Health Monitoring

Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

A Novel Multi-Sensor Hybrid Fusion Framework

Deep Learning for Pixel-Level Image Fusion: Recent Advances and Future Prospects

Learnable fusion mechanisms for multimodal object detection in autonomous vehicles

Multi-Sensor Fusion in Automated Driving: A Survey

DATA FUSION OF ULTRASONIC SENSORS FOR DISTANCE MEASUREMENT

Learning Selective Sensor Fusion for States Estimation

Information Fusion of Multi-Sensor Images

Multimodal Medical Image Fusion: The Perspective of Deep Learning

Enabling Efficient Deep Convolutional Neural Network-based Sensor Fusion for Autonomous Driving

Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles

Multimodal medical image fusion and classification using deep learning techniques

Autonomous Multi-Sensor Fusion Techniques for Environmental Perception in Self-Driving Vehicles