Abstract:Despite the large share of energy consumption, current HVAC systems in buildings fail to meet their primary purpose of maintaining comfortable indoor conditions. Current "one size fits all" approach to control the thermal conditions in an environment lead to a high degree of occupant dissatisfaction. Advancements in Internet of Things and Machine Learning have opened the possibility of deploying different sensors at a wide scale to monitor environmental and physiological information and using collected sensor data to model individual comfort requirements. Thermal imaging has recently gained interest as one of the possible ways to monitor physiological information (skin temperature) for thermal comfort assessment. Previous studies have shown that skin temperatures from different regions of the face, such as forehead, nose, cheeks and ears can provide useful information for predicting thermal sensation at an individual level. However, existing approaches to process thermal images either rely on manual temperature extraction or use methods that are less reliable in accurately identifying different facial regions. One of the major challenges of using thermal imaging for monitoring skin temperatures in actual buildings is that occupants may move relative to the camera. It is not practical to expect building occupants to be oriented facing the cameras at all times, therefore, it is important to be able to extract as much information as possible from instances where it is feasible to extract relevant information. In this paper, we describe an approach to extract skin temperature by locating specific regions of the face in thermal images. The approach involves combining data from RGB images with thermal images and leveraging facial landmark detection in RGB images. We also evaluate our approach with existing approach of face detection used in previous studies. Our study demonstrates that facial landmark detection provides a more accurate calculation of different locations in the face compared to previous studies. We show an improvement in overall quantity and quality of temperature measurements extracted from thermal images compared to previous studies. More accurate temperature measurements from thermal images can improve the accuracy of thermal imaging for modeling and predicting thermal comfort.

Unveiling human eye temperature with deep learning-powered segmentation

Human Body Parsing in Thermal InfraRed Domain

Image Stitching and Partitioning Algorithms for Infrared Thermal Human-Body Images

Inner Eye Canthus Localization for Human Body Temperature Screening

Semantic segmentation of longitudinal thermal images for identification of hot and cool spots in urban areas

Skin Temperature Extraction Using Facial Landmark Detection and Thermal Imaging for Comfort Assessment

ISO-Compatible Personal Temperature Measurement Using Visual and Thermal Images With Facial Region of Interest Detection

Eye Recognition by YOLO for Inner Canthus Temperature Detection in the Elderly Using a Transfer Learning Approach

Convolutional Neural Networks for Semantic Segmentation as a Tool for Multiclass Face Analysis in Thermal Infrared

Thermal Vision: Pioneering Non-Invasive Temperature Tracking in Congested Spaces

Thermal comfort prediction based on automated extraction of skin temperature of face component on thermal image

Machine Learning-Based Automated Thermal Comfort Prediction: Integration of Low-Cost Thermal and Visual Cameras for Higher Accuracy

Algorithm for Automated Segmentation and Feature Extraction of Thermal Images

Investigation of Facial Region Extraction Algorithm Focusing on Temperature Distribution Characteristics of Facial Thermal Images

Self-adversarial Multi-scale Contrastive Learning for Semantic Segmentation of Thermal Facial Images

Segmenting hotspots from medical thermal images using Density-based modified FC-PcFS with spatial information

Influence of Thermal Imagery Resolution on Accuracy of Deep Learning based Face Recognition

Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism

Thermal mapping the eye: A critical review of advances in infrared imaging for disease detection

Mobilenetv3: a deep learning technique for human face expressions identification

The Use of AI for Thermal Emotion Recognition: A Review of Problems and Limitations in Standard Design and Data