Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

A Robust Mixed Error Coding Method Based on Nonconvex Sparse Representation

A Regularized Correntropy Framework for Robust Pattern Recognition

Data representation using robust nonnegative matrix factorization for edge computing

Robust mixed-norm constrained regression with application to face recognitions

Regularized Robust Coding for Face Recognition

Simultaneous image denoising and completion through convolutional sparse representation and nonlocal self-similarity

Robust Nuclear Norm Regularized Regression for Face Recognition with Occlusion

Robust Face Recognition Based on Non-Negative Sparse Discriminative Low-Rank Representation

Sparse fixed-rank representation for robust visual analysis.

Robust Low-Rank Regularized Regression for Face Recognition with Occlusion

Weighted Non-Negative Sparse Low-Rank Representation Classification

Elastic Net with Adaptive Weight for Image Denoising

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Adaptive Structure-constrained Robust Latent Low-Rank Coding for Image Recovery

Robust Face Recognition via Adaptive Sparse Representation

Robust low-rank representation via residual projection for image classification

Robust Subspace Segmentation Via Nonconvex Low Rank Representation.

A locally weighted sparse graph regularized Non-Negative Matrix Factorization method.

Robust Alternating Low-Rank Representation by joint Lp- and L2,p-norm minimization

Group-Based Sparse Representation Based on

Low-Rank 2-D Neighborhood Preserving Projection for Enhanced Robust Image Representation.