Abstract:Over the past few decades, large archives of paper-based historical documents, such as books and newspapers, have been digitized using the Optical Character Recognition (OCR) technology. Unfortunately, this broadly used technology is error-prone, especially when an OCRed document was written hundreds of years ago. Neural networks have shown great success in solving various text processing tasks, including OCR post-correction. The main disadvantage of using neural networks for historical corpora is the lack of sufficiently large training datasets they require to learn from, especially for morphologically rich languages like Hebrew. Moreover, it is not clear what are the optimal structure and values of hyperparameters (predefined parameters) of neural networks for OCR error correction in Hebrew due to its unique features. Furthermore, languages change across genres and periods. These changes may affect the accuracy of OCR post-correction neural network models. To overcome these challenges, we developed a new multi-phase method for generating artificial training datasets with OCR errors and hyperparameters’ optimization for building an effective neural network for OCR post-correction in Hebrew. To evaluate the proposed approach, a series of experiments using several literary Hebrew corpora from various periods and genres were conducted. The obtained results demonstrate that (1) training a network on texts from a similar period dramatically improves the network's ability to fix OCR errors, (2) using the proposed error injection algorithm, based on character-level period-specific errors, minimizes the need for manually corrected data and improves the network accuracy by 9%, (3) the optimized network design improves the accuracy by 3% compared to the state-of-the-art network, and (4) the constructed optimized network outperforms neural machine translation models and industry-leading spellcheckers. The proposed methodology may have practical implications for digital humanities projects that aim to search and analyze OCRed documents in Hebrew and potentially other morphologically rich languages.

What's Wrong with Hebrew NLP? And How to Make it Right

Machine Learning and Hebrew NLP for Automated Assessment of Open-Ended Questions in Biology

A Truly Joint Neural Architecture for Segmentation and Parsing

Multilingual Sequence-to-Sequence Models for Hebrew NLP

Neural Modeling for Named Entities and Morphology (NEMO^2)

Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With

A Novel Challenge Set for Hebrew Morphological Disambiguation and Diacritics Restoration

A Language Modeling Approach to Diacritic-Free Hebrew TTS

HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew

Toward a Period-specific Optimized Neural Network for OCR Error Correction of Historical Hebrew Texts

Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities

Morphological Processing of Low-Resource Languages: Where We Are and What's Next

Natural Language Processing RELIES on Linguistics

Analyzing learner language: the case of the Hebrew Learner Essay Corpus

A Second Wave of UD Hebrew Treebanking and Cross-Domain Parsing

LAF-Fabric: a data analysis tool for Linguistic Annotation Framework with an application to the Hebrew Bible

Towards a General-Purpose Linguistic Annotation Backend

Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles

Nakdan: Professional Hebrew Diacritizer

Building Low-Resource NER Models Using Non-Speaker Annotation