Abstract:The techniques of using neural networks to learn distributed word representations (i.e., word embeddings) have been used to solve a variety of natural language processing tasks. The recently proposed methods, such as CBOW and Skip-gram, have demonstrated their effectiveness in learning word embeddings based on context information such that the obtained word embeddings can capture both semantic and syntactic relationships between words. However, it is quite challenging to produce high-quality word representations for rare or unknown words due to their insufficient context information. In this paper, we propose to leverage morphological knowledge to address this problem. Particularly, we introduce the morphological knowledge as both additional input representation and auxiliary supervision to the neural network framework. As a result, beyond word representations, the proposed neural network model will produce morpheme representations, which can be further employed to infer the representations of rare or unknown words based on their morphological structure. Experiments on an analogical reasoning task and several word similarity tasks have demonstrated the effectiveness of our method in producing high-quality words embeddings compared with the state-of-the-art methods.

An Ensemble Method for Producing Word Representations focusing on the Greek Language

Evaluation of Greek Word Embeddings

Enriching Word Vectors with Subword Information

SubGram: Extending Skip-gram Word Representation with Substrings

The Greek podcast corpus: Competitive speech models for low-resourced languages with weakly supervised data

GREEK-BERT: The Greeks visiting Sesame Street

Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space

Multimodal Skip-gram Using Convolutional Pseudowords

Combining Language and Vision with a Multimodal Skip-gram Model

CIRCE at SemEval-2020 Task 1: Ensembling Context-Free and Context-Dependent Word Representations

PENELOPIE: Enabling Open Information Extraction for the Greek Language through Machine Translation

Co-learning of Word Representations and Morpheme Representations.

Learning Word Representations by Jointly Modeling Syntagmatic and Paradigmatic Relations.

Unsupervised Multimodal Language Representations using Convolutional Autoencoders

Context-Dependent Word Representation for Neural Machine Translation

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Leveraging Human Prior Knowledge to Learn Sense Representations

Multi-phase Word Sense Embedding Learning Using a Corpus and a Lexical Ontology.

Learning Multi-Modal Word Representation Grounded in Visual Context

GreekBART: The First Pretrained Greek Sequence-to-Sequence Model

Word Translation Without Parallel Data