Uncertainty Assessment-Based Active Learning for Reliable Fire Detection Systems
Young-Jin Kim,Won-Tae Kim
DOI: https://doi.org/10.1109/access.2022.3190852
IF: 3.9
2022-07-23
IEEE Access
Abstract:Deep learning technologies, due to their advanced pattern extraction and recognition of high-dimensional data, have been widely adopted into multisensor-based fire detection systems. Since deep learning approaches can generate erroneous predictions due to incomplete training datasets, a retraining process over unseen observations is needed. However, storing a large amount of data from continuous multisensor streams and labeling them to create a retraining dataset are costly and time-consuming. In this paper, we propose an active learning framework based on an informative experience memory that is populated with meaningful retraining data by assessing the uncertainty of the data. In the proposed framework, the deep learning model predicts fire occurrence and estimates model uncertainty by taking advantage of a Bayesian neural network using Monte Carlo dropout. By storing only higher uncertain data points into the fixed-size informative experience memory and querying them to the system managers, the storage and labeling costs are minimized while improving performance. To evaluate our active learning framework with different neural network structures, we develop three Bayesian neural networks based on conventional classification networks, including the feedforward neural network, fully convolutional network, and long short-term memory. We further investigate various uncertainty assessment scoring methods for classification tasks such as entropy, BALD, variation ratios, and mean STD. Experiments on a real dataset show that the Bayesian FCN using the BALD assessment method has the highest performance gain with an F1 score of 0.95, with an improvement of 24% using only 700 data points.
computer science, information systems,telecommunications,engineering, electrical & electronic