Abstract:We present a new four-pronged approach to build firefighter's situational awareness for the first time in the literature. We construct a series of deep learning frameworks built on top of one another to enhance the safety, efficiency, and successful completion of rescue missions conducted by firefighters in emergency first response settings. First, we used a deep Convolutional Neural Network (CNN) system to classify and identify objects of interest from thermal imagery in real-time. Next, we extended this CNN framework for object detection, tracking, segmentation with a Mask RCNN framework, and scene description with a multimodal natural language processing(NLP) framework. Third, we built a deep Q-learning-based agent, immune to stress-induced disorientation and anxiety, capable of making clear navigation decisions based on the observed and stored facts in live-fire environments. Finally, we used a low computational unsupervised learning technique called tensor decomposition to perform meaningful feature extraction for anomaly detection in real-time. With these ad-hoc deep learning structures, we built the artificial intelligence system's backbone for firefighters' situational awareness. To bring the designed system into usage by firefighters, we designed a physical structure where the processed results are used as inputs in the creation of an augmented reality capable of advising firefighters of their location and key features around them, which are vital to the rescue operation at hand, as well as a path planning feature that acts as a virtual guide to assist disoriented first responders in getting back to safety. When combined, these four approaches present a novel approach to information understanding, transfer, and synthesis that could dramatically improve firefighter response and efficacy and reduce life loss.

Real-Time Fire Monitoring and Visualization for the Post-Ignition Fire State in a Building

Towards a Mission-Critical Ambient Intelligent Fire Victims Assistance System

Data-Driven Prediction Methods for Real-Time Indoor Fire Scenario Inferences

Fire and smoke digital twin – A computational framework for modeling fire incident outcomes

Wireless Network for Assessing Temperature Load of Large-Scale Structures Under Fire Hazards

Integrating Deep Learning and Augmented Reality to Enhance Situational Awareness in Firefighting Environments

Generating Synthetic Sensor Data to Facilitate Machine Learning Paradigm for Prediction of Building Fire Hazard

Artificial intelligence assisted IoT-fog based framework for emergency fire response in smart buildings

An early warning AI-powered portable system to reduce workload and inspect environmental damage after natural disasters

Edge computing enabled smart firefighting: opportunities and challenges.

Real-Time Fire Surveillance with Machine Learning Twilio Integration

Fire and Smoke Digital Twin -- A computational framework for modeling fire incident outcomes

An AI-based Early Fire Detection System Utilizing HD Cameras and Real-time Image Analysis

Indoor fire detection utilizing computer vision-based strategies

Enhancing situational awareness with adaptive firefighting drones: leveraging diverse media types and classifiers.

Automatic real-time fire distance, size and power measurement driven by stereo camera and deep learning

A Real-Time Fire Detection Method from Video with Multifeature Fusion

Fire Environment Monitoring Device for Firefighters Based on MSP430F169

Enhancing Situational Awareness with Adaptive Firefighting Drones

A Cyber-Physical System for Wildfire Detection and Firefighting

Towards an Integrated Performance Framework for Fire Science and Management Workflows