Abstract:Fire-related incidents continue to be reported as a leading cause of life and property destruction. Automated fire detection and localization (AFDL) systems have grown in importance with the evolution of applied robotics, especially because use of robots in disaster situations can lead to avoidance of human fatality. The importance of AFDL on resource-constrained devices has further grown, as most unmanned vehicles (drones or ground vehicles) are battery operated with limited computational capacity, the disaster situations cannot guarantee uninterrupted communication with high end resources in the cloud, and yet faster response time is a prime necessity. Traditional computer-vision based techniques require hand-engineered features on a case-by-case basis. Deep Learning -based classifiers perform well for fire/no-fire classification due to the availability of large datasets for training, however, a dearth of good fire localization datasets renders the localization performance below par. We have tried to address both problems with a multi-task learned cascaded model that triggers localization workflow only if the presence of fire is detected, through a strong classifier trained on available large fire datasets. This presents only fire images to a relatively weaker localization model, reducing false positives, false negatives, and thereby improving overall AFDL accuracy. The multi-task learning (MTL) approach for end-to-end training of a stitched classifier and object localizer model on diverse datasets enabled us to build a strong fire classifier and feature extractor. It also resulted in a single unified model, capable of running on “on-board” compute infrastructure without compromising on accuracy. To achieve the target inference rate for the AFDL deployment, we have investigated the effect of quantization and compression due to hardware acceleration on an MTL model. This paper presents an approach to automate the hardware-software co-design to find the optimum parameter partitioning for a given MTL problem, especially when some parts of the model are hardware accelerated. We present combined evaluation results showing that our methodology and the corresponding AFDL model strikes a balance between the frames inferred per second and several accuracy metrics. We report fire localization accuracy in terms of mean average precision (object detection), that is not done earlier for embedded AFDL systems.

DTFL‐DF: Digital twin architecture powered by federated learning decision forest to mitigate fire accidents in mining industry

Heuristic Forest Fire Detection Using the Deep Learning Model with Optimized Cluster Head Selection Technique

Optimizing Federated Learning With Deep Reinforcement Learning for Digital Twin Empowered Industrial IoT

Automated Fire Extinguishing System Using a Deep Learning Based Framework

Accelerated Fire Detection and Localization at Edge

Wireless sensor network assisted automated forest fire detection using deep learning and computer vision model

Decision Support System for the Prediction of Mine Fire Levels in Underground Coal Mining Using Machine Learning Approaches

Digital Twin Applied to Predictive Maintenance for Industry 4.0

Adaptive Federated Learning and Digital Twin for Industrial Internet of Things

A digital twin framework to support vehicle interaction risk management in the mining industry

Generic Multi-Layered Digital-Twin-Framework-Enabled Asset Lifecycle Management for the Sustainable Mining Industry

A digital twin framework for machine learning optimization of aerial fire fighting and pilot safety

Digital Twin Framework for Optimal and Autonomous Decision-Making in Cyber-Physical Systems: Enhancing Reliability and Adaptability in the Oil and Gas Industry

Resource Efficient Asynchronous Federated Learning for Digital Twin Empowered IoT Network

An efficient deep learning architecture for effective fire detection in smart surveillance

Exploring Digital Twin-Based Fault Monitoring: Challenges and Opportunities

A deep learning-enhanced Digital Twin framework for improving safety and reliability in human–robot collaborative manufacturing

TemporalFED: Detecting Cyberattacks in Industrial Time-Series Data Using Decentralized Federated Learning

Digital Twin Based on 3D Visualization and Computer Image Recognition Technology in Coal Preparation Plant

Digital Twin-Assisted Federated Learning with Blockchain in Multi-tier Computing Systems

Sustainable Resource Allocation and Reduce Latency Based on Federated-Learning-Enabled Digital Twin in IoT Devices