Abstract:Heart rate (HR) measurement and monitoring is of great importance to determine the physiological and mental status of individuals. Recently, it has been demonstrated that HR can be remotely retrieved from facial video-based photoplethysmographic signals captured using consumer-grade cameras. However, in existing studies, subjects are mostly required to keep their facial regions of interest (ROIs) within one single camera. To make this technique usable in a daily life situation where subjects move around freely, we launch a preliminary simulated study of seamless remote HR measurement using multiple synchronized cameras by combining ensemble empirical mode decomposition (EEMD) with time-delay canonical correlation analysis (TDCCA), termed as EEMD-TDCCA. At each time point, a target ROI with the largest area is first determined from all the ROIs provided by all the cameras. Then, the RGB time sequence is formed by taking average of all pixels within each target ROI. Afterwards, the green channel time sequence is decomposed into several intrinsic mode functions (IMFs) and only the IMF candidates, whose frequency corresponding to the maximum amplitude falling into the interested HR range will be further processed by TDCCA. Finally, the first pair of the canonical variables having the largest correlation coefficient is the HR source and the corresponding HR is derived by peak detection or frequency analysis. Thirty subjects were recruited and four state-of-the-art methods were employed for comparison. The best performance was achieved by using the proposed EEMD-TDCCA followed by frequency analysis, with the mean absolute error 4.11 bpm, mean percentage error 5.26%, root mean square error 5.37 bpm, the Pearson’s correlation coefficient 0.90 and the intra-class correlation coefficient 0.89, demonstrating the feasibility of our proposed seamless remote HR measurement framework. This study will provide a promising solution for practical and robust non-contact HR measurement applications.

ROI Detection and Tracking for Physiological Monitoring Based on Calibration between RGB-D and Thermal Cameras

Remote Heart Rate Measurement from Face Videos under Realistic Situations

On-the-fly Extrinsic Calibration of Multimodal Sensing System for Fast 3D Thermographic Scanning

Incremental Low Rank Noise Reduction for Robust Infrared Tracking of Body Temperature during Medical Imaging

Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging

Depth and Thermal Sensor Fusion to Enhance 3D Thermographic Reconstruction

RealSense = Real Heart Rate: Illumination Invariant Heart Rate Estimation from Videos

Multi-ROI Spectral Approach for the Continuous Remote Cardio-Respiratory Monitoring from Mobile Device Built-In Cameras

RGB-D Camera based Human Limb Movement Recognition and Tracking in Supine Positions.

Non-Contact Respiration Monitoring and Body Movements Detection for Sleep Using Thermal Imaging

Contactless Measurement of Vital Signs Using Thermal and RGB Cameras: A Study of COVID 19-Related Health Monitoring

Exploring the Feasibility of Seamless Remote Heart Rate Measurement Using Multiple Synchronized Cameras

Accurate Non-Contact Body Temperature Measurement with Thermal Camera under Varying Environment Conditions

Extrinsic Calibration of Thermal Camera and 3D LiDAR Sensor via Human Matching in Both Modalities during Sensor Setup Movement

Robust heart-rate estimation from facial videos using Project_ICA

CalibrationPhys: Self-supervised Video-based Heart and Respiratory Rate Measurements by Calibrating Between Multiple Cameras

Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras

Thermal Vision: Pioneering Non-Invasive Temperature Tracking in Congested Spaces

ISO-Compatible Personal Temperature Measurement Using Visual and Thermal Images With Facial Region of Interest Detection

Skeletal Data Matching and Merging from Multiple RGB-D Sensors for Room-Scale Human Behaviour Tracking