Abstract:<p>Early fire detection is crucial for reducing losses due to fire. Therefore, the development of an efficient fire detection algorithm and alarm system is essential. The image fire detection algorithm is based on the mathematical analysis of images. Current algorithm evaluation methods are not effective and cannot sufficiently distinguish the performance of different detection algorithms. Therefore, such methods are not conducive for evaluating and improving algorithms. Based on a large-scale fire image dataset, the ground-truth complexity of images was quantified according to the time required for humans to detect the presence or absence of fire in the images. Four image complexity metrics based on the characteristics of fire detection are proposed. A comparison of the ground-truth and predicted scores of image complexity revealed that the comprehensive image complexity metric based on the Inception Resnet -v2 predictor was the most effective measurement. Its predicted scores ranked approximately 85% image pairs in the same order as that of the ground-truth complexity scores. Finally, a novel method for evaluating the performance of an image fire detection algorithm based on image complexity is proposed. Evaluation of the performance of five algorithms revealed that the performance of algorithm differs considerably and the proposed method can accurately determine the detection level of the detection algorithm in different image complexity conditions. The results of the study provide a valuable reference for developing and optimizing detection algorithms.</p>

What problem does this paper attempt to address?

The paper primarily investigates the performance evaluation of image fire detection algorithms. Existing evaluation methods cannot effectively distinguish the performance of different detection algorithms, making it difficult to evaluate and improve the algorithms. To address this issue, the authors propose a new evaluation method based on image complexity. Specifically, the study first quantifies the true complexity of images, which is the time required for humans to recognize whether there is a fire in the image. To quantify image complexity, researchers collected a large amount of image data and used a mobile application to obtain the time required by annotators to determine whether there was a fire in the image. In this way, the "true" complexity of each image can be quantified. Next, the research team proposed four image complexity metrics: 1. **Global Background Metrics**: Measures the overall characteristics of the image background, such as brightness and contrast. 2. **Object-to-Background Metrics**: Measures the difference or similarity between the target area and the background area in the image, considering color space and structural information. 3. **Object-to-Disturbance Metrics**: Measures the difference between the target and disturbances, especially when the target and disturbances are visually similar, which may lead to false alarms. 4. **Comprehensive Image Complexity Metrics**: Uses deep learning techniques to automatically extract image features to generate complex image complexity scores. By comparing the correlation between these metrics and the "true" complexity scores, the study found that the comprehensive image complexity metrics based on the Inception Resnet-v2 predictor were the most effective, with a ranking consistency of about 85% with the "true" complexity scores. Finally, the researchers proposed a new method for evaluating image fire detection algorithms based on image complexity. By evaluating five different detection algorithms, the results showed significant differences in the performance of different algorithms. The proposed evaluation method can accurately determine the detection level under different image complexity conditions, providing valuable references for the development and optimization of detection algorithms. In summary, this paper introduces an evaluation method based on image complexity, providing a new perspective for the performance evaluation of image fire detection algorithms, which helps improve the reliability and accuracy of early fire warning systems.

Evaluation of image fire detection algorithms based on image complexity