Abstract:Many remote photoplethysmography (rPPG) estimation models have achieved promising performance in the training domain but often fail to accurately estimate physiological signals or heart rates (HR) in the target domains. Domain generalization (DG) or domain adaptation (DA) techniques are therefore adopted during the offline training stage to adapt the model to either unobserved or observed target domains by utilizing all available source domain data. However, in rPPG estimation problems, the adapted model usually encounters challenges in estimating target data with significant domain variation. In contrast, Test-Time Adaptation (TTA) enables the model to adaptively estimate rPPG signals in various unseen domains by online adapting to unlabeled target data without referring to any source data. In this paper, we first establish a new TTA-rPPG benchmark that encompasses various domain information and HR distributions to simulate the challenges encountered in real-world rPPG estimation. Next, we propose a novel synthetic signal-guided rPPG estimation framework to address the forgetting issue during the TTA stage and to enhance the adaptation capability of the pre-trained rPPG model. To this end, we develop a synthetic signal-guided feature learning method by synthesizing pseudo rPPG signals as pseudo ground truths to guide a conditional generator in generating latent rPPG features. In addition, we design an effective spectral-based entropy minimization technique to encourage the rPPG model to learn new target domain information. Both the generated rPPG features and synthesized rPPG signals prevent the rPPG model from overfitting to target data and forgetting previously acquired knowledge, while also broadly covering various heart rate (HR) distributions. Our extensive experiments on the TTA-rPPG benchmark show that the proposed method achieves superior performance.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the issue of Test-Time Adaptation (TTA) in remote photoplethysmography (rPPG) estimation. Specifically, many existing rPPG estimation models perform well in the training domain but often fail to accurately estimate physiological signals or heart rate (HR) in the target domain. Although Domain Generalization (DG) and Domain Adaptation (DA) techniques can adapt models during the offline training phase by leveraging all available source domain data, these methods still face challenges when dealing with significant domain shifts in target data, such as different video capture settings, varying heart rate ranges, or imbalanced heart rate distributions. In contrast, Test-Time Adaptation (TTA) allows models to adapt online to unlabeled target data without referencing any source data, thereby enabling adaptive rPPG signal estimation across various unseen domains. The main contributions of this paper include: 1. **Establishing a new TTA-rPPG benchmark**: This benchmark covers various domain information and heart rate distributions to simulate the challenges faced in real-world rPPG estimation applications. 2. **Proposing a synthetic signal-guided rPPG estimation framework**: By generating pseudo-rPPG signals as pseudo ground truth, this framework guides a conditional generator to produce latent rPPG features, addressing the forgetting issue during the TTA phase and enhancing the adaptability of pre-trained rPPG models. 3. **Designing an effective spectral entropy minimization technique**: This technique encourages rPPG models to learn new target domain information, preventing the model from overfitting to target data and forgetting previously acquired knowledge. 4. **Extensive experimental validation**: Numerous experiments conducted on the TTA-rPPG benchmark demonstrate that the proposed method outperforms existing DG and DA methods in most benchmark protocols. ### Solutions 1. **Synthetic Signal-Guided Feature Learning**: - **Synthetic rPPG Signals**: Generate pseudo-rPPG signals covering the human heart rate range as pseudo ground truth. - **Generative Feature Learning**: Use a generative module to produce latent rPPG features and optimize the generator parameters through a learning strategy. 2. **Spectral Entropy Minimization**: - Improve the model's adaptability to target data by minimizing the entropy of the power spectral density (PSD) of the estimated rPPG signals. 3. **Feature-Aligned rPPG Estimation**: - Further adapt to the characteristics of target data by aligning target rPPG features with generated rPPG features. 4. **Frequency-Stratified Memory Bank**: - Store generated rPPG signals and features, organized by frequency, for use during the inference phase. ### Experimental Results - **Benchmark Testing**: Constructed a TTA-rPPG benchmark covering multiple publicly available rPPG datasets. - **Performance Comparison**: The proposed method shows significant performance improvements across multiple protocols compared to existing DG and DA methods. - **Ablation Studies**: Validated the effectiveness of different loss terms and optimization strategies, as well as the impact of different feature alignment and memory bank designs. Through the aforementioned methods and experimental validations, this paper successfully addresses the test-time adaptation problem in rPPG estimation, providing new directions for future research.

Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning

Bi-TTA: Bidirectional Test-Time Adapter for Remote Physiological Measurement

Idtl-Rppg: Remote Heart Rate Estimation Using Instance-Based Deep Transfer Learning

SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency

Rppg-Hiba:Hierarchical Balanced Framework for Remote Physiological Measurement

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

Standardized rPPG signal generation based on generative adversarial networks

Robust Test-Time Adaptation in Dynamic Scenarios

Video-based Remote Physiological Measurement via Self-supervised Learning

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Real-Time Heart Rate Estimation Algorithm Based on Adaptive Spectrum Correction and Peak Localization

DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation

Remote physiological signal recovery with efficient spatio-temporal modeling

Facial Video-based Remote Physiological Measurement via Self-supervised Learning

Improved Test-Time Adaptation for Domain Generalization

Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition

Learning Spatio-Temporal Pulse Representation With Global-Local Interaction and Supervision for Remote Prediction of Heart Rate

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

Deep learning-based remote-photoplethysmography measurement from short-time facial video

Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation

Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos