SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency

Yiping Xie,Zitong Yu,Bingjie Wu,Weicheng Xie,Linlin Shen

2024-09-18

Abstract:Remote Photoplethysmography (rPPG) is a non-contact method that uses facial video to predict changes in blood volume, enabling physiological metrics measurement. Traditional rPPG models often struggle with poor generalization capacity in unseen domains. Current solutions to this problem is to improve its generalization in the target domain through Domain Generalization (DG) or Domain Adaptation (DA). However, both traditional methods require access to both source domain data and target domain data, which cannot be implemented in scenarios with limited access to source data, and another issue is the privacy of accessing source domain data. In this paper, we propose the first Source-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which overcomes these limitations by enabling effective domain adaptation without access to source domain data. Our framework incorporates a Three-Branch Spatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency across domains. Furthermore, we propose a new rPPG distribution alignment loss based on the Frequency-domain Wasserstein Distance (FWD), which leverages optimal transport to align power spectrum distributions across domains effectively and further enforces the alignment of the three branches. Extensive cross-domain experiments and ablation studies demonstrate the effectiveness of our proposed method in source-free domain adaptation settings. Our findings highlight the significant contribution of the proposed FWD loss for distributional alignment, providing a valuable reference for future research and applications. The source code is available at <a class="link-external link-https" href="https://github.com/XieYiping66/SFDA-rPPG" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **the problem of poor generalization ability of remote photoplethysmography (rPPG) in unseen domains**. Specifically, traditional rPPG models often perform poorly in the target domain because they have difficulty dealing with the domain gap between different datasets. Current solutions usually rely on domain generalization (DG) or domain adaptation (DA), but these methods require access to data in the source domain and the target domain, which may be limited in practical applications and pose privacy issues.

To solve these problems, the authors propose a **source - free domain adaptation (SFDA) framework**, namely **SFDA - rPPG**, which can perform effective domain adaptation without accessing the source - domain data. This method not only improves the performance of the model on the target domain but also protects the privacy of the source - domain data.

### Specific Problems and Solutions

1. **Problem Description**:
- Traditional rPPG models perform poorly in unseen domains.
- Current domain generalization and domain adaptation methods require access to data in both the source domain and the target domain, which is not feasible in some scenarios and poses privacy risks.

2. **Proposed Solutions**:
- **SFDA - rPPG Framework**: By introducing the source - free domain adaptation method, the model can perform domain adaptation using only the pre - trained model and unlabeled data in the target domain.
- **Three - Branch Spatio - Temporal Consistency Network (TSTC - Net)**: Enhance the consistency of features between different domains.
- **Distribution Alignment Loss Based on Frequency - Domain Wasserstein Distance (FWD)**: Use the optimal transport theory to align the power spectral density distributions between different domains, thereby improving the stability of the model.

### Main Contributions

1. **For the first time, apply source - free domain adaptation to the rPPG field**, ensuring the robustness and privacy protection of the model.
2. **Propose a new frequency - domain Wasserstein distance loss function** for more stably aligning the power spectral density distributions of different frequency bands.
3. **Verify the effectiveness of SFDA - rPPG through cross - dataset experiments**, demonstrating its excellent performance in different domains.

Through these innovations, this paper provides a more general and secure solution for remote physiological measurement technology.