Abstract:Remote photoplethysmography (rPPG), which measures human heart rate without physical contact with the skin, has become active research in recent years. Neural networks have been introduced into rPPG for accurate pulse measurement and have achieved overwhelming results. However, there is a lack of in-depth analysis of key components of neural networks exhibiting a crucial impact on pulse extraction from video. In this paper, we present a network with attention and spatio-temporal convolutional block (ASTNet), exploiting the impact of key factors including different spatio-temporal convolutions, attention mechanism, the number of convolutional layers, and receptive field sizes. The novel attention module named spatio-temporal-channel (STC) attention is designed to jointly learn weights in spatial, temporal, and channel dimensions in a more efficient way. Extensive experiments have been conducted on two uncompressed datasets and one compressed dataset. Results show that ASTNet outperforms state-of-the-art methods in accuracy and computational time. Specifically, networks with larger receptive field sizes and more spatio-temporal blocks generally achieve better performance. Networks with pseudo 3D convolution outperform those with convolutional 3D in static videos, and the opposite is true in motion videos. The results exhibit a similar tendency both on uncompressed and compressed datasets. The proposed method improves the performance of pulse signal compared to PhysNet (the second-best approach in the compared methods), with the signal-to-noise ratio increased by 7.03%, 10.19%, 4.79%, the mean absolute error decreased by 17.95%, 14.17%, 22.76%, and the root-mean-square error decreased by 21.43%, 2.73%, 25.43%, on the PURE, Self-rPPG, and COHFACE datasets, respectively.

Heart Rate Estimation from Facial Videos Based on Convolutional Neural Network

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Remote Heart Rate Measurement from Face Videos under Realistic Situations

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Estimation of vital signs from facial videos via video magnification and deep learning

Remote Photoplethysmography from Low Resolution videos: An end-to-end solution using Efficient ConvNets

Robust Efficient Estimation of Heart Rate Pulse from Video.

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Robust Heart Rate Estimation with Spatial–Temporal Attention Network from Facial Videos

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

Deep learning-based remote-photoplethysmography measurement from short-time facial video

Automated Heartbeat Classification Using 3-D Inputs Based on Convolutional Neural Network with Multi-Fields of View.

A novel one-stage framework for visual pulse rate estimation using deep neural networks

ECG Signal Reconstruction Based on Facial Videos Via Combined Explicit and Implicit Supervision.

Adaptive Heart Rate Estimation from Face Videos

FastBVP-Net: a lightweight pulse extraction network for measuring heart rhythm via facial videos

Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks