Abstract:Abstract This article proposes a comprehensive strategy for training, evaluating, and optimizing domain-specific word2vec-based word embeddings, using social science literature as an example. Our primary objectives are: (1) to train the embeddings utilizing a corpus of social science text, (2) to test their performance against domain-unspecific embeddings using our developed intrinsic and extrinsic evaluation strategy, and (3) to enhance their performance even further by using domain knowledge. As an integral part of this approach, we present SociRel-461 , a domain-knowledge dictionary designed for the intrinsic evaluation and subsequent refinement of social science word embeddings. Using a dataset of 100,000 full-text scientific articles in sociology, we train multiple vector space models, which we then benchmark against a larger, pre-trained general language embedding model as part of our extrinsic evaluation. Furthermore, we developed a transfer learning multi-label classification task for extrinsic evaluation. Our findings reveal that domain-specific embeddings outperform their domain-unspecific counterparts in both intrinsic and extrinsic evaluations. We also investigated the retrofitting post-processing method to enhance domain-unspecific embeddings with the domain knowledge embedded in SociRel-461 . While retrofitting does not enhance our domain-specific vector space models, it significantly improves the performance of the domain-unspecific embeddings. This highlights the potential of retrofitting for the transfer of domain knowledge to domain-unspecific embeddings. Our results emphasize the importance of utilizing domain-specific word embeddings for better performance in domain specific transfer learning tasks, as they outperform conventional embeddings trained on everyday language.

Word embedding analysis on colonial history, present issues, and optimism toward the future in Senegal

The Geometry of Culture: Analyzing Meaning through Word Embeddings

A Primer on Word Embeddings: AI Techniques for Text Analysis in Social Work

Cultural cartography with word embeddings

Simple dynamic word embeddings for mapping perceptions in the public sphere

Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes

Theoretical foundations and limits of word embeddings: what types of meaning can they capture?

Identification, Interpretability, and Bayesian Word Embeddings

Word Embedding for Social Sciences: An Interdisciplinary Survey

More than Bags of Words: Sentiment Analysis with Word Embeddings

Advancing language models through domain knowledge integration: a comprehensive approach to training, evaluation, and optimization of social scientific neural word embeddings

Word Embedding as a Semantic Feature Extraction Technique in Arabic Natural Language Processing: An Overview

A Survey On Neural Word Embeddings

Dialectograms: Machine Learning Differences between Discursive Communities

Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models

AfriVEC: Word Embedding Models for African Languages. Case Study of Fon and Nobiin

Detecting policy preferences and dynamics in the UN general debate with neural word embeddings

Analyzing the Surprising Variability in Word Embedding Stability Across Languages

Word Embedding Techniques for Sentiment Analyzers

Monitoring african geopolitics: a multilingual sentiment and public attention framework

Word embedding and neural network on grammatical gender -- A case study of Swedish