Abstract:Semitic morphologically-rich languages (MRLs) are characterized by extreme word ambiguity. Because most vowels are omitted in standard texts, many of the words are homographs with multiple possible analyses, each with a different pronunciation and different morphosyntactic properties. This ambiguity goes beyond word-sense disambiguation (WSD), and may include token segmentation into multiple word units. Previous research on MRLs claimed that standardly trained pre-trained language models (PLMs) based on word-pieces may not sufficiently capture the internal structure of such tokens in order to distinguish between these analyses. Taking Hebrew as a case study, we investigate the extent to which Hebrew homographs can be disambiguated and analyzed using PLMs. We evaluate all existing models for contextualized Hebrew embeddings on a novel Hebrew homograph challenge sets that we deliver. Our empirical results demonstrate that contemporary Hebrew contextualized embeddings outperform non-contextualized embeddings; and that they are most effective for disambiguating segmentation and morphosyntactic features, less so regarding pure word-sense disambiguation. We show that these embeddings are more effective when the number of word-piece splits is limited, and they are more effective for 2-way and 3-way ambiguities than for 4-way ambiguity. We show that the embeddings are equally effective for homographs of both balanced and skewed distributions, whether calculated as masked or unmasked tokens. Finally, we show that these embeddings are as effective for homograph disambiguation with extensive supervised training as with a few-shot setup.

A Characterwise Windowed Approach to Hebrew Morphological Segmentation

Chinese Word Segmentation Method Based on Dictionary and Frequency of the Words

A Novel Challenge Set for Hebrew Morphological Disambiguation and Diacritics Restoration

Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses?

Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

A realistic and robust model for Chinese word segmentation

A Word Level Segmentation for Off-Line Arabic Characters

A Hybrid Approach to the Real World Text Segmentation

Multilingual Sequence-to-Sequence Models for Hebrew NLP

A Language Modeling Approach to Diacritic-Free Hebrew TTS

Off-Line Arabic Handwritten Words Segmentation using Morphological Operators

A Statistical Approach For Resolving Problematical Word Boundaries In Chinese Lexicography

A Discriminative Latent Variable Chinese Segmenter with Hybrid Word/Character Information.

A Second Wave of UD Hebrew Treebanking and Cross-Domain Parsing

Chinese Word Segmentation without Using Dictionary Based on Unsupervised Learning Strategy

What's Wrong with Hebrew NLP? And How to Make it Right

A morphology-based Chinese word segmentation method

TAMS: Translation-Assisted Morphological Segmentation

A Pragmatic Approach to Increase Accuracy of Chinese Word-Segmentation

A new algorithm for machine printed Arabic character segmentation

On the role of morphology in early spelling in Hebrew and Arabic