Generate, Transform, and Clean: the Role of GANs and Transformers in Palm Leaf Manuscript Generation and Enhancement
Nimol Thuon,Jun Du,Zhenrong Zhang,Jiefeng Ma,Pengfei Hu
DOI: https://doi.org/10.1007/s10032-024-00472-z
2024-01-01
International Journal on Document Analysis and Recognition
Abstract:Palm leaf manuscripts offer a rich source of data critical for document analysis tasks, including character, word, and text analysis. However, their cleaning and denoising present substantial challenges due to language diversity, unbalanced datasets, and inherent variations. Existing research often struggles to overcome these unique conditions, necessitating a novel approach. In response, we introduce the Generate, Transform, and Clean (GTC) strategy, which uses two generative adversarial networks (GANs) for data augmentation and enhancement. First, the “Generate" module utilizes an enhanced deep generative adversarial network (DCGAN) with preprocessing techniques to create more realistic backgrounds for various scripts. Second, the “Transform" module employs a multi-task algorithm for text and background transformation, integrating filtering techniques based on palm leaf characteristics. Lastly, the “Clean" module employs a customized transformer-based conditional generative adversarial network (PALM-GAN) for binarization tasks, enhancing learning quality and stability for palm leaf datasets. For our experiments, we evaluate our approach using palm leaf manuscript datasets, focusing specifically on mixed collections from the ICFHR 2016 and 2018 contests. These experiments include various manuscripts, like Balinese and Sundanese from Indonesia, and Khmer from Cambodia. The results underscore not only the superiority of our proposed GTC approach over existing state-of-the-art methods but also highlight its potential to further advance the digitization processes for historical palm leaf documents.