2C2S: A two-channel and two-stream transformer based framework for offline signature verification

Jian-Xin Ren,Yu-Jie Xiong,Hongjian Zhan,Bo Huang

DOI: https://doi.org/10.1016/j.engappai.2022.105639

IF: 8

2022-11-30

Engineering Applications of Artificial Intelligence

Abstract:Recently, with the outstanding performance of the transformer in NLP, approaches that employ the transformer to address vision problem is becoming a research focus. However, transformer-based research rarely focuses on signature verification. To fill this gap, this paper proposes a two-channel and two-stream transformer approach (2C2S) to cope with the signature verification problem. 2C2S is composed of original and central streams. The original stream receives the original signature pair as input, and the central stream receives the signature pair generated by cropping the central at the original pair as input. In order to establish the associations among feature channels, a squeeze-and-excitation operation is applied between two standard Swin Transformer blocks. Moreover, an up-sampling enhancement module directly steers the model to focus on useful information. The verification accuracy of 2C2S on SUES-SiG and several publically available datasets: CEDAR, BHSig-B, and BHSig-H, reaches 93.25%, 90.68%, 100%, and 72.22%, respectively. Extensive experiments illustrate that the proposed framework is competitive with the existing techniques for offline handwritten signature verification.

automation & control systems,computer science, artificial intelligence,engineering, electrical & electronic, multidisciplinary

What problem does this paper attempt to address?

The paper aims to address the problem of offline handwritten signature verification. Specifically:

1. **Proposing a New Framework**: The paper proposes a two-channel dual-stream (2C2S) framework based on the Transformer structure for offline handwritten signature verification. This framework efficiently captures stroke information using the attention mechanism.

2. **Feature Channel Association**: The squeeze-and-excitation (SE) operation is introduced to establish connections between feature channels between two standard Swin Transformer blocks.

3. **Upsampling Enhancement Module**: An upsampling enhancement module is proposed to guide the model to focus on important features and ignore redundant information.

Through these methods, the paper aims to improve the accuracy of offline signature verification, especially in handling high-forgery signatures. Experimental results show that this method outperforms existing techniques on multiple public datasets (such as SUES-SiG, CEDAR, BHSig-B, and BHSig-H).