Abstract:Fire disasters pose a significant threat to human safety. Therefore, timely and effective fire detection is crucial for mitigating these threats. Combining visible light and thermal infrared for multimodal flame detection can fully utilize the visual and temperature distribution information of flames, potentially considerably enhancing the accuracy and robustness of flame detection methods. This approach is a highly promising detection method. However, the visible light and thermal infrared modalities differ fundamentally in imaging principles, pixel resolution, and texture information. Thus, effective fusion of these modalities becomes challenging. To address this issue, a novel flame detection method that integrates visible light and thermal infrared images is introduced. For the visible light modality, an overall model based on Mask R-CNN is designed, with ConvNeXt as the backbone, FPN as the neck, and a cascade structure as the detection head. Then, for the thermal infrared modality, to adapt to its weak semantic and strong texture features, we specifically modified the model's neck to better extract the underlying texture information of the image using the PAFPN structure. Furthermore, we designed a multimodal fusion algorithm using GIoU to fuse detection information from the visible light and thermal infrared modalities to address the weak alignment of detection targets in imaging principles, pixel resolution, and texture information. Experimental results on both public and self-collected datasets demonstrate that our proposed method outperforms other mainstream target detection networks in flame detection. Moreover, ablation experiments suggest that multimodal fusion significantly improves the overall performance of the algorithm. Specifically, our method achieves an accuracy of 85.33, a recall of 99.33, and an F1 score of 90.03.

Vision Based Flame Detection Using Compressed Domain Motion Prediction and Multi-Feature Fusion.

Real-time multi-feature based fire flame detection in video.

Flame Detection in Surveillance

A Fire-Alarming Method Based on Video Processing

Deep Learning Based Fire Detection System For Surveillance Videos

Multi-Feature Based Fire Detection in Video

A Real-Time Fire Detection Method from Video with Multifeature Fusion

A Fast Video Fire Detection of Irregular Burning Feature in Fire-Flame Using in Indoor Fire Sensing Robots

A Method to Distinguish the Fire and Flickering Vehicle Light

An Early Detection Algorithm for Small Flame Based on Vision System with Variant Field of View

Weakly Aligned Multimodal Flame Detection for Fire-Fighting Robots

Multi-Scale Video Flame Detection for Early Fire Warning Based on Deep Learning

Towards a solid solution of real-time fire and flame detection

Fire Video Image Detection Based on a Convolutional Neural Network

Flame Detection Using Deep Learning

An efficient fire detection algorithm based on multi‐scale convolutional neural network

A Flame Localization Method Combing Visible-Light and Near-Infrared Cameras

A Study on Flame Detection Method Combining Visible Light and Thermal Infrared Multimodal Images

Video Fire Detection Methods Based on Deep Learning: Datasets, Methods, and Future Directions

A Feasible System of Automatic Flame Detection and Tracking for Fire-fighting Robot

A Real-Time Flame Detection Method Using Deformable Object Detection and Time Sequence Analysis