Tingzhang Luo,Yichao Liu,Yuanyuan Liu,Andi Zhang,Xin Wang,Yibing Zhan,Chang Tang,Leyuan Liu,Zhe Chen
Abstract:We introduce a novel task, Generalized Facial Expression Category Discovery (G-FACE), that discovers new, unseen facial expressions while recognizing known categories effectively. Even though there are generalized category discovery methods for natural images, they show compromised performance on G-FACE. We identified two biases that affect the learning: implicit bias, coming from an underlying distributional gap between new categories in unlabeled data and known categories in labeled data, and explicit bias, coming from shifted preference on explicit visual facial change characteristics from known expressions to unknown expressions. By addressing the challenges caused by both biases, we propose a Debiased G-FACE method, namely DIG-FACE, that facilitates the debiasing of both implicit and explicit biases. In the implicit debiasing process of DIG-FACE, we devise a novel learning strategy that aims at estimating and minimizing the upper bound of implicit bias. In the explicit debiasing process, we optimize the model's ability to handle nuanced visual facial expression data by introducing a hierarchical category-discrimination refinement strategy: sample-level, triplet-level, and distribution-level optimizations. Extensive experiments demonstrate that our DIG-FACE significantly enhances recognition accuracy for both known and new categories, setting a first-of-its-kind standard for the task.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is a new task in Facial Expression Recognition (FER) - Generalized Facial Expression Category Discovery (G - FACE). Specifically, G - FACE aims to simultaneously recognize known facial expression categories and discover new, unseen facial expression categories. Existing methods encounter two main problems when dealing with this task:
1. **Implicit Bias**: Due to the distribution difference between new categories in unlabeled data and known categories in labeled data, the decision boundary learned by the model has a subtle change, which increases the learning difficulty.
2. **Explicit Bias**: The slight visual feature preferences between known and unknown expressions are different, making the learned decision boundary blurry and difficult to distinguish.
To solve these problems, the authors propose a new de - biasing method - DIG - FACE, which improves the robustness and accuracy of the model on the G - FACE task through implicit and explicit de - biasing strategies.
### Specific Problem Description
1. **Implicit Bias**: In the semi - supervised learning process, new categories contained in unlabeled data will cause a subtle change in the decision boundary, resulting in a distribution gap. Although this bias cannot be directly observed from the image data, it can be constrained by mathematical techniques. For this reason, the authors introduce the F - discrepancy metric to estimate and minimize the upper bound of the implicit bias.
2. **Explicit Bias**: Explicit bias is manifested as a slight visual feature overlap between known and unknown categories, resulting in a blurry decision boundary. To meet this challenge, the authors propose a hierarchical category discriminative optimization strategy, including sample - level, triplet - level, and distribution - level optimization, to enhance the recognition ability of known and unknown categories.
### DIG - FACE Framework
- **Implicit De - biasing Stage**: Estimate and minimize the maximum bias by maintaining the consistency of the main classification head and the auxiliary classification head on the labeled data and maintaining the inconsistency on the unlabeled data.
- **Explicit De - biasing Stage**: Enhance the recognition of known and unknown categories through sample - level, triplet - level, and distribution - level optimization, and improve the decision - making through parameter learning.
### Experimental Results
The experimental results show that DIG - FACE significantly improves the recognition accuracy of known and new categories on multiple datasets, especially on datasets such as RAF - DB, FerPlus, and AffectNet. This proves the effectiveness and superiority of DIG - FACE in the G - FACE task.
In summary, this paper solves the implicit and explicit bias problems in generalized facial expression category discovery by introducing the DIG - FACE framework, thereby significantly improving the performance of the model in recognizing known and discovering new categories.