Abstract:This paper presents a large and systematic assessment of distributional models for Galician. To this end, we have first trained and evaluated static word embeddings (e.g., word2vec , GloVe), and then compared their performance with that of current contextualised representations generated by neural language models. First, we have compiled and processed a large corpus for Galician, and created four datasets for word analogies and concept categorisation based on standard resources for other languages. Using the aforementioned corpus, we have trained 760 static vector space models which vary in their input representations (e.g., adjacency-based versus dependency-based approaches), learning algorithms, size of the surrounding contexts, and in the number of vector dimensions. These models have been evaluated both intrinsically, using the newly created datasets, and on extrinsic tasks, namely on POS-tagging, dependency parsing, and named entity recognition. The results provide new insights into the performance of different vector models in Galician, and about the impact of several training parameters on each task. In general, fastText embeddings are the static representations with the best performance in the intrinsic evaluations and in named entity recognition, while syntax-based embeddings achieve the highest results in POS-tagging and dependency parsing, indicating that there is no significant correlation between the performance in the intrinsic and extrinsic tasks. Finally, we have compared the performance of static vector representations with that of BERT-based word embeddings, whose fine-tuning obtains the best performance on named entity recognition. This comparison provides a comprehensive state-of-the-art of current models in Galician, and releases new transformer-based models for NER. All the resources used in this research are freely available to the community, and the best models have been incorporated into SemantiGal, an online tool to explore vector representations for Galician.

Applying Distributional Compositional Categorical Models of Meaning to Language Translation

Experimental Support for a Categorical Compositional Distributional Model of Meaning

Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics

Compositional Operators in Distributional Semantics

A Categorical Compositional Distributional Modelling for the Language of Life

Local similarity and global variability characterize the semantic space of human languages

Using Language Models to Disambiguate Lexical Choices in Translation

Unsupervised Image Translation with Distributional Semantics Awareness

The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations

Emerging Language Spaces Learned From Massively Multilingual Corpora

Cross-Dialect Sentence Transformation: A Comparative Analysis of Language Models for Adapting Sentences to British English

How do lexical semantics affect translation? An empirical study

Machine Translation of Low-Resource Indo-European Languages

A Computational Approach to Measuring the Semantic Divergence of Cognates

On Evaluating Multilingual Compositional Generalization with Translated Datasets

Training and evaluation of vector models for Galician

A Mathematical Model for Universal Semantics

Transformers for Low-Resource Languages:Is Féidir Linn!

Multimodal Distributional Semantics

Re-Representing Metaphor: Modeling Metaphor Perception Using Dynamically Contextual Distributional Semantics

Contextual Text Understanding in Distributional Semantic Space.