Abstract:If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100% random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem.

Deep Features for Efficient Multi-Biometric Recognition with Face and Ear Images

A feature-level fusion based improved multimodal biometric recognition system using ear and profile face

Feature Fusion Method Based On Kcca For Ear And Profile Face Based Multimodal Recognition

Convolutional Neural Network Approach Based on Multimodal Biometric System with Fusion of Face and Finger Vein Features

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Enhanced multimodal biometric recognition systems based on deep learning and traditional methods in smart environments

Multimodal Recognition Based on Fusion of Ear and Profile Face

Ear and Face Based Multimodal Recognition Based on Kfda

MULTI-MODALITY RECOGNITION OF HUMAN FACE AND EAR BASED ON DEEP LEARNING

Multimodal Recognition Using Face and Ear

Deep Hybrid Multimodal Biometric Recognition System Based on Features-Level Deep Fusion of Five Biometric Traits

Multi-Dictionary Based Collaborative Representation: with Applications to 3D Ear and 3D Palmprint Identification

Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identification

Feature Fusion Method Based on KCCA and Multi-Modality Recognition Fusing Ear and Face Profile Information

Multimodal Approach for Enhancing Biometric Authentication

The Study of Feature-Level Fusion Algorithm for Multimodal Recognition

A Novel Hybrid Multibiometrics Based on the Fusion of Dual Iris, Visible and Thermal Face Images

Research on Kernel-Based Feature Fusion Algorithm in Multimodal Recognition

The Study of Multimodal Recognition Based on Ear and Face

Multimodal Feature-Level Fusion for Biometrics Identification System on IoMT Platform.

Multimodal Ear Recognition Based on 2D+3D Feature Fusion.