Abstract:Fire disasters pose a significant threat to human safety. Therefore, timely and effective fire detection is crucial for mitigating these threats. Combining visible light and thermal infrared for multimodal flame detection can fully utilize the visual and temperature distribution information of flames, potentially considerably enhancing the accuracy and robustness of flame detection methods. This approach is a highly promising detection method. However, the visible light and thermal infrared modalities differ fundamentally in imaging principles, pixel resolution, and texture information. Thus, effective fusion of these modalities becomes challenging. To address this issue, a novel flame detection method that integrates visible light and thermal infrared images is introduced. For the visible light modality, an overall model based on Mask R-CNN is designed, with ConvNeXt as the backbone, FPN as the neck, and a cascade structure as the detection head. Then, for the thermal infrared modality, to adapt to its weak semantic and strong texture features, we specifically modified the model's neck to better extract the underlying texture information of the image using the PAFPN structure. Furthermore, we designed a multimodal fusion algorithm using GIoU to fuse detection information from the visible light and thermal infrared modalities to address the weak alignment of detection targets in imaging principles, pixel resolution, and texture information. Experimental results on both public and self-collected datasets demonstrate that our proposed method outperforms other mainstream target detection networks in flame detection. Moreover, ablation experiments suggest that multimodal fusion significantly improves the overall performance of the algorithm. Specifically, our method achieves an accuracy of 85.33, a recall of 99.33, and an F1 score of 90.03.

A Flame Localization Method Combing Visible-Light and Near-Infrared Cameras

Vision Based Flame Detection Using Compressed Domain Motion Prediction and Multi-Feature Fusion.

Level Set Based Flame Segmentation

A Method to Distinguish the Fire and Flickering Vehicle Light

A Study on Flame Detection Method Combining Visible Light and Thermal Infrared Multimodal Images

Flame Detection in Surveillance

Deep Learning Based Fire Detection System For Surveillance Videos

Real-time multi-feature based fire flame detection in video.

A Fire-Alarming Method Based on Video Processing

Towards a solid solution of real-time fire and flame detection

A Real-Time Fire Detection Method from Video with Multifeature Fusion

An Early Detection Algorithm for Small Flame Based on Vision System with Variant Field of View

Weakly Aligned Multimodal Flame Detection for Fire-Fighting Robots

Object Detection through Fires Using Violet Illumination Coupled with Deep Learning

A Feasible System of Automatic Flame Detection and Tracking for Fire-fighting Robot

Automatic real-time fire distance, size and power measurement driven by stereo camera and deep learning

A Fire Source Localization Algorithm Based on Temperature and Smoke Sensor Data Fusion

Multi-Feature Based Fire Detection in Video

A combined real-time intelligent fire detection and forecasting approach through cameras based on computer vision method

A Fast Video Fire Detection of Irregular Burning Feature in Fire-Flame Using in Indoor Fire Sensing Robots

An Early Flame Detection System Based on Image Block Threshold Selection Using Knowledge of Local and Global Feature Analysis