Handwritten Chinese signature detection with simple Copy–Paste augmentation on power plants technical documents
Ying Zhang,Jian Zhang,Kaihong Yan,Hongwei Wang,Gaoang Wang
DOI: https://doi.org/10.1007/s11761-023-00374-8
2023-01-01
Service Oriented Computing and Applications
Abstract:Verification and validation of signed documents holds the key to successful operation in many industries such as manufacturing and energy. In power plants, a huge amount of technical documents is accumulated over years, and the manual verification of these documents is highly challenging. The rapid development of deep learning makes automatic verification a possible solution for this challenging task. In this paper, we divide the process of signature verification into signature detection, restoration, and verification procedures to streamline automatic verification. Specifically, signature detection is the first key part in which we take advantage of recent developments in the fast-developing object detection field. Unlike other object detection tasks, signature detection faces the great challenge of little publicly available dataset and considerable research is still needed. In this paper, we explore Copy–Paste augmentation to address the data scarcity problem. In our experiments, we find that the Copy–Paste augmentation effectively improves the mAP@0.5:0.95 of the detection models. In addition, we conduct a Signature-Date experiment to prove that Copy–Paste augmentation allows the detection of handwritten signatures in complex situations in which they are signed closely with handwritten dates underneath and is difficult for detectors to distinguish without further annotation. As for detection models, we explore three different object detection models, namely faster R-CNN, YOLOv5, and DETR, which represent two-stage, one-stage, and transformer-based models, respectively. We find that YOLOv5 with Copy–Paste augmentation outperforms the other models. To evaluate our research, we conduct an ablation study using different training schemes and training data, which further validates our method.