Abstract:The goal of information security is to prevent unauthorized access to data. There are several conventional ways to confirm user identity, such as using a password, user name, and keys. These conventional methods are rather limited; they can be stolen, lost, copied, or cracked. Because multimodal biometric identification systems are more secure and have higher recognition efficiency than unimodal biometric systems, they get attention. Single-modal biometric recognition systems perform poorly in real-world public security operations because of poor biometric data quality. Some of the drawbacks of current multimodal fusion methods include low generalization and single-level fusion. This study presents a novel multimodal biometric fusion model that significantly enhances accuracy and generalization through the power of artificial intelligence. Various fusion methods, encompassing pixel-level, feature-level, and score-level fusion, are seamlessly integrated through deep neural networks. At the pixel level, we employ spatial, channel, and intensity fusion strategies to optimize the fusion process. On the feature level, modality-specific branches and jointly optimized representation layers establish robust dependencies between modalities through backpropagation. Finally, intelligent fusion techniques, such as Rank-1 and modality evaluation, are harnessed to blend matching scores on the score level. To validate the model's effectiveness, we construct a virtual homogeneous multimodal dataset using simulated operational data. Experimental results showcase significant improvements compared to single-modal algorithms, with a remarkable 2.2 percentage point increase in accuracy achieved through multimodal feature fusion. The score fusion method surpasses single-modal algorithms by 3.5 percentage points, reaching an impressive retrieval accuracy of 99.6%.

Multi-Modal Person Identification In A Smart Environment

A Multimodal Sensor Fusion Framework Robust to Missing Modalities for Person Recognition

Comparative Analysis of Modality Fusion Approaches for Audio-Visual Person Identification and Verification

Efficient Multimodal Fusion for Person Authentication Combining Three Biometric Modalities

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Audio-Visual Fusion Based on Interactive Attention for Person Verification

Multimodal Biometrics Using Multiple Feature Representations to Speaker Identification System

Human identification and verification based on multimodal fusion of face and ear

Multimodal person authentication using speech, face and visual speech

Audio-visual multi-person tracking and identification for smart environments

LRMM: Low Rank Multi-Scale Multi-Modal Fusion for Person Re-Identification Based on RGB-NI-TI

Multimodal biometric scheme for human authentication technique based on voice and face recognition fusion

Large-scale Multi-modal Person Identification in Real Unconstrained Environments

ISL Person Identification Systems in the CLEAR 2007 Evaluations

Multimodal information fusion for selected multimedia applications

Enhancing Recognition in Multimodal Biometric Systems: Score Normalization and Fusion of Online Signatures and Fingerprints

Multimodal Feature-Level Fusion for Biometrics Identification System on IoMT Platform.

Artificial intelligence-Enabled deep learning model for multimodal biometric fusion

Interact, Embed, and EnlargE: Boosting Modality-Specific Representations for Multi-Modal Person Re-identification

Multi-biometrics Fusion Based on Dynamic Weighting of Characteristic Signal Quality

Multimodal Biometric Fusion Approach Based on Iris and Face