Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

Two-stage Patch-Based Sparse Multi-Value Descriptor for Face Recognition

Two-stage nonnegative sparse representation for large-scale face recognition.

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Face Recognition Based on Non-local Similarity Dictionary.

Face Recognition based on scale invariant feature transform and Spatial Pyramid Representation

Local Spatial Continuity Steered Sparse Representation for Occluded Face Recognition

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

A Multi-modal 2D + 3D Face Recognition Method with a Novel Local Feature Descriptor

Discriminative Patch-Based Sparse Representation for Face Recognition

Face Recognition Using Data Driven Local Appearance Features

Face Recognition Based on Multi-Wavelet and Sparse Representation.

Sparse Spatiotemporal Descriptor for Micro-Expression Recognition Using Enhanced Local Cube Binary Pattern

Robust 3D Face Recognition by Local Shape Difference Boosting

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Automatic Two Phase Sparse Representation Method and Face Recognition Experiments.

A Multi-Modal Face Recognition Method Using Complete Local Derivative Patterns And Depth Maps

Two Phase Representation Discriminant Projection for Face Recognition

Multi-scale Patch Based Representation Feature Learning for Low-Resolution Face Recognition

Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition

A Multi-Phase Sparse Probability Framework Via Entropy Minimization for Single Sample Face Recognition

Patch Based Collaborative Representation with Gabor Feature and Measurement Matrix for Face Recognition