Facial Demorphing via Identity Preserving Image Decomposition

Nitish Shukla,Arun Ross

2024-08-21

Abstract:A face morph is created by combining the face images usually pertaining to two distinct identities. The goal is to generate an image that can be matched with two identities thereby undermining the security of a face recognition system. To deal with this problem, several morph attack detection techniques have been developed. But these methods do not extract any information about the underlying bonafides used to create them. Demorphing addresses this limitation. However, current demorphing techniques are mostly reference-based, i.e, they need an image of one of the identities to recover the other. In this work, we treat demorphing as an ill-posed decomposition problem. We propose a novel method that is reference-free and recovers the bonafides with high accuracy. Our method decomposes the morph into several identity-preserving feature components. A merger network then weighs and combines these components to recover the bonafides. Our method is observed to reconstruct high-quality bonafides in terms of definition and fidelity. Experiments on the CASIA-WebFace, SMDD and AMSL datasets demonstrate the effectiveness of our method.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the paper attempts to solve

This paper aims to solve the security problems brought by face morphs. Specifically, a face morph is generated by combining facial images of two or more different identities. Such an image can be matched to multiple identities, thus weakening the security of the face recognition system. To address this issue, researchers have developed a variety of Morph Attack Detection (MAD) techniques. However, these methods are usually unable to extract the real - identity information used to create the morph image.

This paper proposes a new deconstruction method to deal with the demorphing problem of face morphs. Its main contributions are as follows:

1. **Reference - independent facial deconstruction**: Most of the existing deconstruction methods require a reference image to recover another identity, while the method in this paper is reference - independent and can directly recover the original identity image from the morph image.
2. **High - precision identity recovery**: This method decomposes the morph image into multiple identity - preserving feature components and performs a weighted combination of these components through a merger network to reconstruct the original identity image with high fidelity and high definition.
3. **Flexibility and generalization ability**: Through multi - component deconstruction and automatic selection of important components, this method has higher flexibility and better generalization ability when dealing with face morphs.

### Method overview

The method proposed in this paper includes two main stages:
- **Deconstruction stage**: Use a decomposer network to decompose the input facial image into multiple "incomprehensible" components, ensuring that each component does not contain any visual information about the facial identity.
- **Reconstruction stage**: Use a merger network to receive these components and combine them in sequence, and finally generate an image highly similar to the original facial image.

For the deconstruction of face morphs, the modified pipeline can decompose the input morph image into multiple components containing partial information and select these components through the merger network to reconstruct the original identity image.

### Loss function

To train this pipeline, the author designed a loss function \( L_{\text{decomp}} \), which consists of two parts:
- The first part encourages the network to generate an output image that is visually similar to the input image.
- The second part penalizes the similarity between the input image and the components, ensuring that no visual information about the input face is retained in the components and that there is no redundant information between the components.

In addition, a cross - road loss is introduced to ensure that the model correctly pairs the output image with the real - identity image during the training process.

### Experimental results

Experiments show that this method has achieved excellent results on the CASIA - WebFace, SMDD, and AMSL datasets, especially in the deconstruction and reconstruction of face morphs. Specifically:
- The overall matching accuracy on the CASIA - WebFace dataset is 96.03%, and the accuracy is 99.67% after removing the images with undetected faces.
- The matching accuracy on the SMDD dataset is 97.23%, and the accuracy is 98.89% after removing the images with undetected faces.
- For the deconstruction of face morphs, this method can generate a reconstructed image highly similar to the real - identity image, and the recovery accuracy reaches 99.84% and 99.56% on the AMSL dataset, and 97.80% and 99.37% on the SMDD dataset.

In conclusion, the method proposed in this paper performs well in the deconstruction and reconstruction of face morphs and has potential application value, especially in the field of biometric security.