Abstract:When compared to unimodal systems, multimodal biometric systems have several advantages, including lower error rate, higher accuracy, and larger population coverage. However, multimodal systems have an increased demand for integrity and privacy because they must store multiple biometric traits associated with each user. In this paper, we present a deep learning framework for feature-level fusion that generates a secure multimodal template from each user's face and iris biometrics. We integrate a deep hashing (binarization) technique into the fusion architecture to generate a robust binary multimodal shared latent representation. Further, we employ a hybrid secure architecture by combining cancelable biometrics with secure sketch techniques and integrate it with a deep hashing framework, which makes it computationally prohibitive to forge a combination of multiple biometrics that pass the authentication. The efficacy of the proposed approach is shown using a multimodal database of face and iris and it is observed that the matching performance is improved due to the fusion of multiple biometrics. Furthermore, the proposed approach also provides cancelability and unlinkability of the templates along with improved privacy of the biometric data. Additionally, we also test the proposed hashing function for an image retrieval application using a benchmark dataset. The main goal of this paper is to develop a method for integrating multimodal fusion, deep hashing, and biometric security, with an emphasis on structural data from modalities like face and iris. The proposed approach is in no way a general biometric security framework that can be applied to all biometric modalities, as further research is needed to extend the proposed framework to other unconstrained biometric modalities.

A Secure Visual-thermal Fused Face Recognition System Based on Non-Linear Hashing

A Fusion Framework For Face Recognition Under Varying Illumination Based On Multi-Scale Analysis

Deep CNN based binary hash video representations for face retrieval.

Enhancing thermal facial recognition leveraging large datasets and hybrid algorithms

Face Anti-Spoofing Via Jointly Modeling Local Texture and Constructed Depth

Nonlinear fusion for face recognition using fuzzy integral

A Bayesian Hashing Approach and Its Application to Face Recognition.

FRIH: A face recognition framework using image hashing

Near infrared face recognition using Zernike moments and Hermite kernels

A Synthesis-Based Approach for Thermal-to-Visible Face Verification

Fusion of Visible and Infrared Imagery for Face Recognition

An enhanced thermal face recognition method based on multiscale complex fusion for Gabor coefficients

Spontaneous Facial Expression Recognition by Using Feature-Level Fusion of Visible and Thermal Infrared Images

Privacy-Preserving Face Recognition in Hybrid Frequency-Color Domain

Optimal Bayesian Hashing for Efficient Face Recognition

Depth Image Hashing Algorithm Based on Local Global Feature Fusion

Expression Recognition from Visible Images with the Help of Thermal Images

A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network

A Machine Learning-Based Secure Face Verification Scheme and Its Applications to Digital Surveillance

Deep Hashing for Secure Multimodal Biometrics

Linear and Nonlinear Feature-Based Fusion Algorithms for Face Recognition.