Abstract:We evaluate 8 different word embedding models on their usefulness for predicting the neural activation patterns associated with concrete nouns. The models we consider include an experiential model, based on crowd-sourced association data, several popular neural and distributional models, and a model that reflects the syntactic context of words (based on dependency parses). Our goal is to assess the cognitive plausibility of these various embedding models, and understand how we can further improve our methods for interpreting brain imaging data. We show that neural word embedding models exhibit superior performance on the tasks we consider, beating experiential word representation model. The syntactically informed model gives the overall best performance when predicting brain activation patterns from word embeddings; whereas the GloVe distributional method gives the overall best performance when predicting in the reverse direction (words vectors from brain images). Interestingly, however, the error patterns of these different models are markedly different. This may support the idea that the brain uses different systems for processing different kinds of words. Moreover, we suggest that taking the relative strengths of different embedding models into account will lead to better models of the brain activity associated with words.

Good, Better, Best: Choosing Word Embedding Context

Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings

Learning Context-Specific Word/Character Embeddings.

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Redefining Context Windows for Word Embedding Models: An Experimental Study

Learning word representation by jointly using neighbor and syntactic contexts

Different Contexts Lead to Different Word Embeddings

Learning word embeddings from dependency relations

Learning Effective Word Embedding Using Morphological Word Similarity

Context Aware Document Embedding.

Learning Word Embedding with Better Distance Weighting and Window Size Scheduling

Dependency-based Convolutional Neural Networks for Sentence Embedding

Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection

Category Enhanced Word Embedding.

Learning Semantic Representations for Novel Words: Leveraging Both Form and Context

Learning Word Embeddings from Intrinsic and Extrinsic Views

Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity

Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction

Learning Context-Sensitive Word Embeddings with Neural Tensor Skip-Gram Model

Embedding Word Similarity with Neural Machine Translation

Topical Word Embeddings