Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

Ear recognition method based on fusion features of global and local features

A new facial feature based on the fusion of texture and shape characteristics

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

An Efficient Approach of 3D Ear Recognition

Face Recognition Method Based on Combination Features of Global and Local Features

Ear Identification by Fusion of Segmented Slice Regions using Invariant Features: An Experimental Manifold with Dual Fusion Approach

SIFT-based Ear Recognition by Fusion of Detected Keypoints from Color Similarity Slice Regions

Human Ear Image Recognition Method Using PCA and Fisherface Complementary Double Feature Extraction

3D Ear Recognition Using Local Salience and Principal Manifold

Ear Recognition With Score-Level Fusion Based On CMC In Long-Wave Infrared Spectrum

Face Recognition Method of Multiple Features Local and Global Fusion

A new method of feature fusion and its application in image recognition

Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition

A feature-level fusion based improved multimodal biometric recognition system using ear and profile face

Face Recognition Based on Curvelets and Local Binary Pattern Features Via Using Local Property Preservation

Integration of global and local feature for face recognition

Human ear print recognition based on fusion of difference theoretic texture and gradient direction pattern features

A Fast 3D Ear Recognition Method Based on Local Surface Patch

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Facial Target Detection and Recognition Algorithm Based on Information Fusion

A New Gabor Phase Difference Pattern for Face and Ear Recognition