Abstract:Fire disasters pose a significant threat to human safety. Therefore, timely and effective fire detection is crucial for mitigating these threats. Combining visible light and thermal infrared for multimodal flame detection can fully utilize the visual and temperature distribution information of flames, potentially considerably enhancing the accuracy and robustness of flame detection methods. This approach is a highly promising detection method. However, the visible light and thermal infrared modalities differ fundamentally in imaging principles, pixel resolution, and texture information. Thus, effective fusion of these modalities becomes challenging. To address this issue, a novel flame detection method that integrates visible light and thermal infrared images is introduced. For the visible light modality, an overall model based on Mask R-CNN is designed, with ConvNeXt as the backbone, FPN as the neck, and a cascade structure as the detection head. Then, for the thermal infrared modality, to adapt to its weak semantic and strong texture features, we specifically modified the model's neck to better extract the underlying texture information of the image using the PAFPN structure. Furthermore, we designed a multimodal fusion algorithm using GIoU to fuse detection information from the visible light and thermal infrared modalities to address the weak alignment of detection targets in imaging principles, pixel resolution, and texture information. Experimental results on both public and self-collected datasets demonstrate that our proposed method outperforms other mainstream target detection networks in flame detection. Moreover, ablation experiments suggest that multimodal fusion significantly improves the overall performance of the algorithm. Specifically, our method achieves an accuracy of 85.33, a recall of 99.33, and an F1 score of 90.03.

Multimodal Fusion Induced Attention Network for Industrial VOCs Detection

Multi-spectral Image Fusion for Moving Object Detection

IVJDN: an End-to-End Network for Joint Infrared and Visible Image Fusion and Detection

Multi-Modality Image Fusion and Object Detection Based on Semantic Information

Registration-Free Hybrid Learning Empowers Simple Multimodal Imaging System for High-quality Fusion Detection

Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

A Study on Flame Detection Method Combining Visible Light and Thermal Infrared Multimodal Images

Multiscale Channel Attention Network for Infrared and Visible Image Fusion.

An object detection algorithm based on infrared-visible dual modal feature fusion

A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism

Multi-scale Receptive Field Grouped and Split Attention Network for Real-time Detection of Hazardous Chemical Gas Leaks in Infrared Images

MAFusion: Multiscale Attention Network for Infrared and Visible Image Fusion

Cross-Modality Attentive Feature Fusion for Object Detection in Multispectral Remote Sensing Imagery

A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion

ACDF-YOLO: Attentive and Cross-Differential Fusion Network for Multimodal Remote Sensing Object Detection

HATF: Multi-Modal Feature Learning for Infrared and Visible Image Fusion via Hybrid Attention Transformer

VOCs Leakage Detection Based on Weak Temporal Attention Asymmetric 3D Convolution

Multimodal Industrial Anomaly Detection via Hybrid Fusion

Background-Aware Cross-Attention Multiscale Fusion for Multispectral Object Detection

Airborne Small Target Detection Method Based on Multimodal and Adaptive Feature Fusion

Advancing infrared and visible image fusion with an enhanced multiscale encoder and attention-based networks