MMHSV: A Multimodal Handwritten Signature Verification Fusing Dynamic and Static Feature.

Qixiang Li,Zhaoya Wang,Lianwen Jin,Nurbiya Yadikar,Kurban Ubul
DOI: https://doi.org/10.1109/ICASSP48485.2024.10447071
2024-01-01
Abstract:In recent years, significant progress has been made in the field of handwritten signature verification through methods based on deep learning. However, due to the high intra-class variability and high inter-class similarity of signature samples, achieving high accuracy and security in handwritten signature verification systems remains challenging. In this paper, we propose a deep learning-based multimodal handwritten signature verification framework, MMHSV, to fuse dynamic and static features by utilizing the complementary signal components of signature images and pen stroke sounds, and explore the feasibility of multimodal signature verification. MMHSV consists of an innovative joint-embedded feature representation method based on multi-task learning and a dual-path network model for signature representation extraction. To support our method, we have curated a multimodal signature dataset, serving as a benchmark for the proposed technique. Preliminary findings suggest that our approach offers a pioneering solution in the realm of handwritten signature verification and security metrics that surpass the benchmarks set by the unimodal state-of-the-art methods.
What problem does this paper attempt to address?