Abstract:Physical signs of the body that are too subtle to be observed by human eyes can reflect significant health indicators. Although many vision-based approaches have been devoted to recovery, they topically focus on recognizing explicit features such as colors, textures, and patches, and pay less attention to the entanglement and disentanglement among implicit biological characteristics. Meanwhile, existing deep networks for remote physiological detection are generally weak or deliberately neglectful in eliminating long-term time-varying interference and noise. To address these issues, we propose TranPulse, a novel remote estimation paradigm dedicated to video transformer-based polyphysiological disentanglement for robust heart rate (HR) prediction. Specifically, we improve existing single-stage transformer-based cardiac estimation backbones into a targeted two-stage architecture, which consists of a pair of asymmetric encoder and decoder. The encoder, which is the important module for heart waveform prediction, guides global attention to spatio-temporal enhancement and perception of periodic signals according to facial frame differences. The decoder, which is the core strategy pipeline for unwrapping time-varying disturbances, uses waveform gradient variations as the constraint for high-dimensional representations to separate spikes from multiple uncorrelated obstructions. Simultaneously, we integrate the synchronous computation of the encoder's fusion of appearance and frame difference to provide more detailed guidance for in-the-wild spatio-temporal modeling, and design a more reliable regression loss function to coordinate long-term temporal and frame-wise spatial supervision. We train, validate, and practice our proposed model on multiple publicly available datasets, where it achieves competitive performance in pulse estimation by extensive experimental results.

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

Non-Contact Heart Rate Estimation from Photoplethysmography Using EEMD and Convolution-Transformer Network

TranPhys: Spatiotemporal Masked Transformer Steered Remote Photoplethysmography Estimation

PulseNet: A multitask learning network for remote heart rate estimation

STFMRPPGFormer:Vision Transformer Network Based on MLP-3D and STFM for Remote Heart Rate Measurement

TransPPG: Two-stream Transformer for Remote Heart Rate Estimate

TCNTransNet: A Semi-Supervised Temporal-Spatial Fusion Framework for Heart Rate Estimation from Camera Video

CDCCA-RPPGFormer: Transformer-Like Network Based on 3D-CDC-ST and ECA for Remote Heart Rate Measurement

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

TransPhys: Transformer-based unsupervised contrastive learning for remote heart rate measurement

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Rppg-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

TranPulse: Remote Photoplethysmography Estimation with Time-Varying Supervision to Disentangle Multi-Physiologically Interference

Information-enhanced Network for Noncontact Heart Rate Estimation from Facial Videos

Remote Heart Rate Measurement from Face Videos under Realistic Situations

PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer

Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild

Learning Spatio-Temporal Pulse Representation With Global-Local Interaction and Supervision for Remote Prediction of Heart Rate

ETA-rPPGNet: Effective Time-Domain Attention Network for Remote Heart Rate Measurement.

Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos