Abstract:Models of word meaning that exploit patterns of word usage across large text corpora to capture semantic relations, like the topic model and word2vec, condense word-by-context co-occurrence statistics to induce representations that organize words along semantically relevant dimensions (e.g., synonymy, antonymy, hyponymy, etc.). However, their reliance on latent representations leaves them vulnerable to interference, makes them slow learners, and commits to a dual-systems account of episodic and semantic memory. We show how it is possible to construct the meaning of words online during retrieval to avoid these limitations. We implement a spreading activation account of word meaning in an associative net, a one-layer highly recurrent network of associations, called a Dynamic-Eigen-Net, that we developed to address the limitations of earlier variants of associative nets when scaling up to deal with unstructured input domains like natural language text. We show that spreading activation using a one-hot coded Dynamic-Eigen-Net outperforms the topic model and reaches similar levels of performance as word2vec when predicting human free associations and word similarity ratings. Latent Semantic Analysis vectors reached similar levels of performance when constructed by applying dimensionality reduction to the Shifted Positive Pointwise Mutual Information but showed poorer predictability for free associations when using an entropy-based normalization. An analysis of the rate at which the Dynamic-Eigen-Net reaches asymptotic performance shows that it learns faster than word2vec. We argue in favor of the Dynamic-Eigen-Net as a fast learner, with a single-store, that is not subject to catastrophic interference. We present it as an alternative to instance models when delegating the induction of latent relationships to process assumptions instead of assumptions about representation.

Growing Networks – Modelling the Growth of Word Association Networks for Hungarian and English

Growth Model for Complex Networks with Hierarchical and Modular Structures.

Deep Learning-Based Analysis of Ancient Greek Literary Texts in English Version: A Statistical Model Based on Word Frequency and Noise Probability for the Classification of Texts

Networks with Growth and Preferential Attachment: Modeling and Applications

Approaching Human Language with Complex Networks.

Language Clustering with Word Co-Occurrence Networks Based on Parallel Texts

Developing neural machine translation models for Hungarian-English

Markov Random Geometric Graph (MRGG): A Growth Model for Temporal Dynamic Networks

Towards hypergraph cognitive networks as feature-rich models of knowledge

Polish and English wordnets -- statistical analysis of interconnected networks

Latent Relations at Steady-state with Associative Nets

Learning the mechanisms of network growth

A Computational Model to Disentangle Semantic Information Embedded in Word Association Norms

Modelling Populations of Interaction Networks via Distance Metrics

Geographies of an Online Social Network

Action selection in growing state spaces: Control of Network Structure Growth

Modeling texts with networks: comparing five approaches to sentence representation

Co-Evolution of Multi-Typed Objects in Dynamic Star Networks

GraphInterpreter: a visual analytics approach for dynamic networks evolution exploration via topic models

Revisiting Link Prediction: Evolving Models and Real Data Findings