Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

Hybrid Classification Approach Using Extreme Learning Machine and Sparse Representation Classifier with Adaptive Threshold

Fast Sparse Approximation of Extreme Learning Machine

Image super-resolution representation via image patches based on extreme learning machine

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

Extreme Learning Machine Based Transfer Learning for Data Classification.

A Fast Incremental Method Based on Regularized Extreme Learning Machine

Multiclass AdaBoost ELM and Its Application in LBP Based Face Recognition

Multi-class AdaBoost ELM

Extreme Learning Machine Combining Hidden-Layer Feature Weighting and Batch Training for Classification

Correntropy Extreme Learning Machine Based on Spatial Pyramid Matching and Local Receptive Field

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

An Extreme Learning Machine Based on Artificial Immune System.

An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine

An extreme learning machine model based on adaptive multi-fusion chaotic sparrow search algorithm for regression and classification

A New Supervised Image Classification Using Data Set Partition

${{\Rm E}^{2}}{\rm LMs}$ : Ensemble Extreme Learning Machines for Hyperspectral Image Classification

AdaBoost-assisted Extreme Learning Machine for Efficient Online Sequential Classification

Kernel Extreme Learning Machine Optimized by the Sparrow Search Algorithm for Hyperspectral Image Classification

Hyperspectral Remote Sensing Image Classification with Information Discriminative Extreme Learning Machine

A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning Machine

Sparse Bayesian ELM Handling with Missing Data for Multi-class Classification