Abstract:Fire disasters pose a significant threat to human safety. Therefore, timely and effective fire detection is crucial for mitigating these threats. Combining visible light and thermal infrared for multimodal flame detection can fully utilize the visual and temperature distribution information of flames, potentially considerably enhancing the accuracy and robustness of flame detection methods. This approach is a highly promising detection method. However, the visible light and thermal infrared modalities differ fundamentally in imaging principles, pixel resolution, and texture information. Thus, effective fusion of these modalities becomes challenging. To address this issue, a novel flame detection method that integrates visible light and thermal infrared images is introduced. For the visible light modality, an overall model based on Mask R-CNN is designed, with ConvNeXt as the backbone, FPN as the neck, and a cascade structure as the detection head. Then, for the thermal infrared modality, to adapt to its weak semantic and strong texture features, we specifically modified the model's neck to better extract the underlying texture information of the image using the PAFPN structure. Furthermore, we designed a multimodal fusion algorithm using GIoU to fuse detection information from the visible light and thermal infrared modalities to address the weak alignment of detection targets in imaging principles, pixel resolution, and texture information. Experimental results on both public and self-collected datasets demonstrate that our proposed method outperforms other mainstream target detection networks in flame detection. Moreover, ablation experiments suggest that multimodal fusion significantly improves the overall performance of the algorithm. Specifically, our method achieves an accuracy of 85.33, a recall of 99.33, and an F1 score of 90.03.

A Study on a Complex Flame and Smoke Detection Method Using Computer Vision Detection and Convolutional Neural Network

Deep Learning Based Fire Detection System For Surveillance Videos

Vision Based Flame Detection Using Compressed Domain Motion Prediction and Multi-Feature Fusion.

Fire Video Image Detection Based on a Convolutional Neural Network

A convolutional neural network-based flame detection method in video sequence

Video Smoke Detection Base on Dense Optical Flow and Convolutional Neural Network.

Joint Detection of Smoke and Flame in Photovoltaic System Based on Deep Learning

An Integrated Smoke Detection Method Based on Convolutional Neural Network and Image Processing

Flame Detection Using Deep Learning

Video Flame Detection Method Based on Two-Stream Convolutional Neural Network

Improving Fire Detection Accuracy through Enhanced Convolutional Neural Networks and Contour Techniques

A Study on Flame Detection Method Combining Visible Light and Thermal Infrared Multimodal Images

Real-time Video Fire Smoke Detection by Utilizing Spatial-Temporal ConvNet Features.

Information-Guided Flame Detection Based on Faster R-CNN

Forest Fire Smoke Recognition Based on Convolutional Neural Network

Video Fire Recognition Based on Multi-Channel Convolutional Neural Network

Smoke Detection Based on Dark Channel and Convolutional Neural Networks

A Two-Stream CNN Model with Adaptive Adjustment of Receptive Field Dedicated to Flame Region Detection

Video Fire Smoke Detection Using Motion and Color Features

A Novel Algorithm for Fire/Smoke Detection Based on Computer-Vision

Deep Learning-Based Fire Detection for Enhanced Safety Systems