Abstract:Innovative engineering solutions that are efficient, quick, and simple to use are crucial given the rapid industrialization and technology breakthroughs in Industry 5.0. One of the areas receiving attention is the rise in gas leakage accidents at coal mines, chemical companies, and home appliances. To prevent harm to both the environment and human lives, rapid and automated detection and identification of the gas type is necessary. Most of the previous studies used a single mode of data to perform the detection process. However, instead of using a single source/mode, multimodal sensor fusion offers more accurate results. Furthermore, the majority used individual feature extraction approaches that extract either spatial or temporal information. This paper proposes a deep learning-based (DL) pipeline to combine multimodal data acquired via infrared (IR) thermal imaging and an array of seven metal oxide semiconductor (MOX) sensors forming an electronic nose (E-nose). The proposed pipeline is based on three convolutional neural networks (CNNs) models for feature extraction and bidirectional long-short memory (Bi-LSTM) for gas detection. Two multimodal data fusion approaches are used, including intermediate and multitask fusion. Discrete wavelet transform (DWT) is utilized in the intermediate fusion to combine the spatial features extracted from each CNN, providing spectral–temporal representation. In contrast, in multitask fusion, the discrete cosine transform (DCT) is used to merge all of the features obtained from the three CNNs trained with the multimodal data. The results show that the proposed fusion approach has boosted the gas detection performance reaching an accuracy of 98.47% and 99.25% for intermediate and multitask fusion, respectively. These results indicate that multitask fusion is superior to intermediate fusion. Therefore, the proposed system is capable of detecting gas leakage accurately and could be used in industrial applications.

A Multimodal Fusion-Based LNG Detection for Monitoring Energy Facilities (Student Abstract)

Multidimensional Information Fusion and Broad Learning System-Based Condition Recognition for Energy Pipeline Safety

Tensor-Based Approach for Liquefied Natural Gas Leakage Detection From Surveillance Thermal Cameras: A Feasibility Study in Rural Areas

Flame Detection in Surveillance

Multitask Deep Learning-Based Pipeline for Gas Leakage Detection via E-Nose and Thermal Imaging Multimodal Fusion

Distributed Optical Fiber Intrusion Warning Based on Multi-model Fusion

Infrared and Visual Fusion Detection Framework for Oil and Gas Pipeline Environmental Inspection

Multimodal Fusion Induced Attention Network for Industrial VOCs Detection

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

A Cross-modal Fusion Method for Multispectral Small Ship Detection

Adaptive multimodal feature fusion with frequency domain gate for remote sensing object detection

Multi-modal Information Fusion for LiDAR-based 3D Object Detection Framework

Lightweight Transformer for Multi-Modal Object Detection (Student Abstract)

Registration-Free Hybrid Learning Empowers Simple Multimodal Imaging System for High-quality Fusion Detection

LMDFusion: A lightweight infrared and visible image fusion network for substation equipment based on mask and residual dense connection

A Study on Flame Detection Method Combining Visible Light and Thermal Infrared Multimodal Images

Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

Mask-Guided Mamba Fusion for Drone-Based Visible-Infrared Vehicle Detection

MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detection

Research on Active Intelligent Perception Technology of Vessel Situation Based on Multisensor Fusion

AI-Based Energy Transportation Safety: Pipeline Radial Threat Estimation Using Intelligent Sensing System