SREDS: A dichromatic separation based measure of skin color

Keivan Bahmani,Richard Plesh,Chinmay Sahu,Mahesh Banavar,Stephanie Schuckers

DOI: https://doi.org/10.48550/arXiv.2104.02926

2021-04-07

Abstract:Face recognition (FR) systems are fast becoming ubiquitous. However, differential performance among certain demographics was identified in several widely used FR models. The skin tone of the subject is an important factor in addressing the differential performance. Previous work has used modeling methods to propose skin tone measures of subjects across different illuminations or utilized subjective labels of skin color and demographic information. However, such models heavily rely on consistent background and lighting for calibration, or utilize labeled datasets, which are time-consuming to generate or are unavailable. In this work, we have developed a novel and data-driven skin color measure capable of accurately representing subjects' skin tone from a single image, without requiring a consistent background or illumination. Our measure leverages the dichromatic reflection model in RGB space to decompose skin patches into diffuse and specular bases.

Image and Video Processing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the performance differences among different groups of people in facial recognition systems, especially the recognition accuracy issues caused by skin color variations. Specifically:

1. **Performance differences in facial recognition systems**: Currently, many facial recognition models show different performances among different groups of people. Especially among people with darker skin tones, this difference is more obvious. This is mainly because skin color variations can affect the light reflection characteristics of images, thus affecting the accuracy of facial recognition.

2. **Limitations of existing methods**: Existing skin color measurement methods (such as ITA and RSR) rely on consistent background and lighting conditions or require labeled datasets, and these conditions are difficult to meet in practical applications. In addition, these methods are sensitive to lighting changes and cannot remain stable in complex environments.

To solve these problems, the author proposes a new skin color measurement method - Skin Reflection Estimation based on Dichromatic Reflection Separation (SREDS). This method aims to provide a data - driven skin color measurement method that is not limited by background and lighting conditions by decomposing the specular and diffuse reflection components of skin reflection. The main advantages of SREDS are:

- **No need for consistent background and lighting conditions**: SREDS can accurately measure skin color in natural environments without strict background and lighting control.
- **Insensitive to lighting changes**: By extracting the diffuse reflection component of the skin, SREDS can reduce the impact of lighting changes on skin color measurement.
- **Data - driven method**: SREDS uses techniques such as Non - negative Matrix Factorization (NMF) to extract useful features from skin images, making the measurement results more reliable and highly interpretable.

In summary, the goal of this paper is to develop a method that can stably and accurately measure skin color under different lighting conditions to improve the performance differences of facial recognition systems among different groups of people.