Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

Using score normalization to solve the score variation problem in face authentication

Score Normalization for Demographic Fairness in Face Recognition

Score normalization in multimodal biometric systems

Adaptive Illumination Normalization Via Adaptive Illumination Preprocessing and Modified Weber-Face

Enhancing Recognition in Multimodal Biometric Systems: Score Normalization and Fusion of Online Signatures and Fingerprints

Multimodal biometric authentication based on score level fusion of finger biometrics

Optimal weight of the score level multiplebiometric feature fusion

Score Normalization for Text-Independent Speaker Verification Systems

Score domain speaking rate normalization for speaker recognition

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Score Normalization-Based Speaking-Style Variation Robust Speaker Recognition

Multimodal Biometric Authentication Based on Score Level Fusion Using Support Vector Machine

Face Pose Normalization and Simulation Methods Based on Multi-View Face Alignment

Score Level Fusion Scheme Based on Adaptive Local Gabor Features for Face-Iris-Fingerprint Multimodal Biometric

Patch-Wise Normalization for Pose-Invariant Face Recognition from Single Sample

Fusing face and fingerprint for identity authentication by SVM

Information Fusion in Face and Fingerprint Identity Verification System

FaceScore: Benchmarking and Enhancing Face Quality in Human Generation

An efficient illumination normalization method for face recognition

Continuous Pose Normalization for Pose-Robust Face Recognition

Multi-Biometrics Fusion for Identity Verification