Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

Unimodal biometric system based on local topology structure preserving projections

Unimodal Biometric System Based on Subspace Learning Method

Multimodal biometrics using geometry preserving projections

Multimodal Biometric Identification System Based on Finger Geometry, Knuckle Print and Palm Print

Face Recognition Based on Local Binary Pattern and Improved Pairwise-constrained Multiple Metric Learning

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Bimodal Biometrics Based on a Representation and Recognition Approach

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Face Recognition Based on Curvelets and Local Binary Pattern Features Via Using Local Property Preservation

Uncorrelated Multilinear Geometry Preserving Projections For Multimodal Biometrics Recognition

The Biometric Recognition on Contactless Multi-Spectrum Finger Images

A Multi-Modal Face Recognition Method Using Complete Local Derivative Patterns And Depth Maps

Sparse Tensor Discriminative Locality Alignment For Gait Recognition

Multi-modal biometric recognition algorithm based on iris and facial images

A face recognition algorithm using local binary patterns and locality preserving projections

A Unified 3D Face Authentication Framework Based on Robust Local Mesh SIFT Feature

A Vision System for Multi-View Face Recognition

NSCTWavelet: An efficient method for multimodal biometric recognition based on pixel level fusion

A hybrid fusion model of iris, palm vein and finger vein for multi-biometric recognition system

A Modified Chaotic Binary Particle Swarm Optimization Scheme and Its Application in Face-Iris Multimodal Biometric Identification

A Robust Single-sensor Face and Iris Biometric Identification System Based on Multimodal Feature Extraction Network