Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

A Novel Race Classification Method Based on Periocular Features Fusion

A new facial feature based on the fusion of texture and shape characteristics

Accurate Iris Center Localization Method Using Facial Landmark, Snakuscule, Circle Fitting and Binary Connected Component

An ICA-Based Other-Race Effect Elimination for Facial Expression Recognition.

A Method for Accurate Localization of Facial Features

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Robust 3D Face Recognition by Local Shape Difference Boosting

Race classification method based on cascade voting mechanism from video

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Feature Fusion for Facial Landmark Point Location

A Facial Feature Discovery Framework for Race Classification Using Deep Learning

Learning and Fusing Multimodal Features from and for Multi-task Facial Computing

Feature Level Analysis for 3D Facial Expression Recognition.

Score Level Fusion Scheme Based on Adaptive Local Gabor Features for Face-Iris-Fingerprint Multimodal Biometric

A Novel Facial Feature Localization and Extraction Method

A Deep Feature Fusion Network Based on Multiple Attention Mechanisms for Joint Iris-Periocular Biometric Recognition

Gender Classification via Global-Local Features Fusion

Novel Approaches To Improve Iris Recognition System Performance Based On Local Quality Evaluation And Feature Fusion

A Multi-modal 2D + 3D Face Recognition Method with a Novel Local Feature Descriptor

Near-infrared and visible light image fusion algorithm for face recognition

Exploiting Effective Facial Patches for Robust Gender Recognition