Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

A Robust Single-sensor Face and Iris Biometric Identification System Based on Multimodal Feature Extraction Network

Iris-Face Fusion and Security Analysis Based on Fisher Discriminant

Multibiometrics Fusion For Identity Authentication: Dual Iris, Visible And Thermal Face Imagery

Multimodal Biometric Fusion Approach Based on Iris and Face

Multimodal Feature-Level Fusion for Biometrics Identification System on IoMT Platform.

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

A Novel Hybrid Multibiometrics Based on the Fusion of Dual Iris, Visible and Thermal Face Images

Multi-modal biometric recognition algorithm based on iris and facial images

Face-Iris Multimodal Biometric Scheme Based on Feature Level Fusion

Face and Iris Fusion Recognition Based on Adaboost

A hybrid fusion model of iris, palm vein and finger vein for multi-biometric recognition system

A Deep Feature Fusion Network Based on Multiple Attention Mechanisms for Joint Iris-Periocular Biometric Recognition

Study on Multi-Biometric Feature Fusion and Recognition Model

Joint Iris and Facial Recognition Based on Feature Fusion and Biomimetic Pattern Recognition

A novel multibiometric template security scheme for the fusion of dual iris, visible and thermal face images

Robust Multimodal Biometric Authentication Integrating Iris, Face and Palmprint

Multibiometrics Using a Single Face Image

A Multimodal Biometric Authentication Method Based on Feature Level Fusion

Multimodal Eye Biometric System Based on Contour Based E-CNN and Multi Algorithmic Feature Extraction Using SVBF Matching

Score Level Fusion Scheme Based on Adaptive Local Gabor Features for Face-Iris-Fingerprint Multimodal Biometric

Recognition algorithm for multimodal biometric fusion based on hand vein and iris