Continual Learning for Remote Physiological Measurement: Minimize Forgetting and Simplify Inference

Qian Liang,Yan Chen,Yang Hu

2024-07-19

Abstract:Remote photoplethysmography (rPPG) has gained significant attention in recent years for its ability to extract physiological signals from facial videos. While existing rPPG measurement methods have shown satisfactory performance in intra-dataset and cross-dataset scenarios, they often overlook the incremental learning scenario, where training data is presented sequentially, resulting in the issue of catastrophic forgetting. Meanwhile, most existing class incremental learning approaches are unsuitable for rPPG measurement. In this paper, we present a novel method named ADDP to tackle continual learning for rPPG measurement. We first employ adapter to efficiently finetune the model on new tasks. Then we design domain prototypes that are more applicable to rPPG signal regression than commonly used class prototypes. Based on these prototypes, we propose a feature augmentation strategy to consolidate the past knowledge and an inference simplification strategy to convert potentially forgotten tasks into familiar ones for the model. To evaluate ADDP and enable fair comparisons, we create the first continual learning protocol for rPPG measurement. Comprehensive experiments demonstrate the effectiveness of our method for rPPG continual learning. Source code is available at \url{<a class="link-external link-https" href="https://github.com/MayYoY/rPPGDIL" rel="external noopener nofollow">this https URL</a>}

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the catastrophic forgetting problem in remote physiological measurement (rPPG). Specifically, existing methods perform poorly in handling incremental learning scenarios. Especially when facing the domain distribution differences between different datasets, the model is prone to forget the knowledge learned previously, resulting in a decline in performance.

### Problem Background

Remote photoplethysmography (rPPG) extracts physiological signals (such as heart rate and heart rate variability) by analyzing color changes in facial videos. Although existing rPPG methods perform well in single - dataset or cross - dataset scenarios, they usually ignore incremental learning scenarios. In such scenarios, training data is provided sequentially, which will lead to the catastrophic forgetting problem, that is, the model forgets the knowledge of old tasks when learning new tasks.

### Research Objectives

The goal of the paper is to propose a new method to solve the incremental learning problem in rPPG measurement, especially how to minimize catastrophic forgetting and simplify the inference process. To this end, the authors introduce a method named ADDP, which is based on adapter fine - tuning and domain prototypes to address these issues.

### Main Contributions

1. **First Exploration**: As far as the authors know, this is the first domain - incremental - learning research for rPPG measurement.
2. **Innovative Methods**:
- Use an adapter (Adapter) to fine - tune the pre - trained model to improve the stability of the model.
- Design domain prototypes (Domain Prototypes), including style prototypes and noise prototypes, for feature enhancement and knowledge consolidation.
- Propose a prototype - based inference simplification strategy to transform unfamiliar tasks into forms that the model is more familiar with.
3. **Establish a Benchmark**: Create the first domain - incremental - learning benchmark for rPPG measurement and conduct extensive experimental verification.

### Method Overview

- **Adapter Fine - Tuning**: Use the adapter module to efficiently fine - tune the frozen backbone network, and design a differential normalization (DiffNorm) module to effectively fuse the appearance and dynamic features of the input video.
- **Prototype Enhancement**: Extract the domain prototypes (style and noise) of old tasks and use them to enhance samples during the training of new tasks to consolidate past knowledge.
- **Inference Simplification**: Use the most familiar style prototype to transform test samples, enabling the model to handle inference problems more efficiently.

Through these strategies, the ADDP method can effectively alleviate the catastrophic forgetting problem during the incremental learning process and improve the generalization ability of the model.