Toward real text manipulation detection: New dataset and new solution
Dongliang Luo,Yuliang Liu,Rui Yang,Xianjin Liu,Jishen Zeng,Yu Zhou,Xiang Bai
DOI: https://doi.org/10.1016/j.patcog.2024.110828
IF: 8
2024-08-19
Pattern Recognition
Abstract:With the surge in realistic text tampering, detecting fraudulent text in images has gained prominence for maintaining information security. However, the high costs associated with professional text manipulation and annotation limit the availability of real-world datasets. With most relying on synthetic tampering, they inadequately replicate real-world tampering attributes. To address this issue, we present the Real Text Manipulation (RTM) dataset, encompassing 9,000 text images, which include 6,000 manually tampered images, created using a variety of techniques, alongside 3,000 unaltered text images for evaluating solution stability. Our evaluations indicate that existing methods falter in text forgery detection on the RTM dataset. We propose a robust baseline solution featuring a Consistency-aware Aggregation Hub and a Gated Cross Neighborhood-attention Fusion module for efficient multi-modal information fusion, supplemented by a Tampered-Authentic Contrastive Learning module during training, enriching feature representation distinction. This framework, extendable to other dual-stream architectures, demonstrated notable localization performance improvements of 3.99% and 5.76% on IoU and F1-measure, respectively. Our contributions aim to propel advancements in real-world text tampering detection. Code and dataset will be made available at https://github.com/DrLuo/RTM .
computer science, artificial intelligence,engineering, electrical & electronic