Abstract:Modern face recognition systems utilize deep neural networks to extract salient features from a face. These features denote embeddings in latent space and are often stored as templates in a face recognition system. These embeddings are susceptible to data leakage and, in some cases, can even be used to reconstruct the original face image. To prevent compromising identities, template protection schemes are commonly employed. However, these schemes may still not prevent the leakage of soft biometric information such as age, gender and race. To alleviate this issue, we propose a novel technique that combines Fully Homomorphic Encryption (FHE) with an existing template protection scheme known as PolyProtect. We show that the embeddings can be compressed and encrypted using FHE and transformed into a secure PolyProtect template using polynomial transformation, for additional protection. We demonstrate the efficacy of the proposed approach through extensive experiments on multiple datasets. Our proposed approach ensures irreversibility and unlinkability, effectively preventing the leakage of soft biometric attributes from face embeddings without compromising recognition accuracy.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to solve the problem of privacy leakage in modern face recognition systems, especially the risk of leaking soft biometric information (such as age, gender, and race) from face embeddings. Specifically:
1. **Privacy risks of face embeddings**: Modern face recognition systems use deep neural networks to extract features from face images and store these features as templates (embeddings). These embeddings are vulnerable to data leakage and may even be used to reconstruct the original face image.
2. **Limitations of existing protection schemes**: Although existing template protection schemes (such as PolyProtect) can provide a certain level of protection, they still cannot completely prevent the leakage of soft biometric information.
3. **Combining fully homomorphic encryption (FHE) with PolyProtect**: To further enhance privacy protection, the author proposes a new technique that combines fully homomorphic encryption (FHE) and the PolyProtect template protection scheme. Through this method, face embeddings can be processed in an encrypted state, ensuring that only authorized parties can decrypt and access the data, thereby effectively preventing the leakage of soft biometric information.
4. **Maintaining recognition accuracy**: While protecting privacy, this method also ensures that the accuracy of face recognition is not significantly affected.
### Main contributions
- **Innovative privacy protection mechanism**: By combining FHE with PolyProtect, a novel framework is proposed that can maintain the accuracy of face recognition while protecting privacy.
- **Experimental verification**: Through extensive experiments on multiple datasets, the effectiveness of the proposed method is proven, demonstrating its superior performance in preventing the leakage of soft biometric information.
- **Theoretical guarantee**: FHE provides strict theoretical guarantees, ensuring the security and irreversibility of face embeddings.
### Formula representation
When describing algorithms and experimental results, some key formulas involved in the paper are as follows:
- **Polynomial transformation**:
\[
p_1 = c_1 v_1^{e_1} + c_2 v_2^{e_2} + \ldots + c_m v_m^{e_m}
\]
where \( p_1 \) is an element in the PolyProtect template, \( V = [v_1, v_2,..., v_n] \) is the original n - dimensional face embedding, and \( C = [c_1, c_2,..., c_m] \) and \( E = [e_1, e_2,..., e_m] \) are user - defined coefficients and exponents respectively.
- **Privacy gain (Privacy Gain)**:
\[
PG = (1 - R_p) - (1 - R_o)
\]
where \( R_o \) and \( R_p \) represent the recognition performance of the original data and the privacy - enhanced data respectively.
- **Suppression rate (Suppression Rate)**:
\[
SR = \frac{A_o - A_p}{A_o}
\]
where \( A_o \) and \( A_p \) represent the attribute prediction accuracy in the cases of non - privacy - enhanced and privacy - enhanced respectively.
Through these methods and techniques, this paper successfully solves the problem of privacy leakage in face recognition systems, especially making significant progress in the protection of soft biometric information.