Abstract:Under a theory of event representations that defines events as dynamic changes in objects across both time and space, as in the proposal of Intersecting Object Histories (Altmann & Ekves, 2019), the encoding of changes in state is a fundamental first step in building richer representations of events. In other words, there is an inherent dynamic that is captured by our knowledge of events. In the present study, we evaluated the degree to which this dynamic was inferable from just the linguistic signal, without access to visual, sensory, and embodied experience, using recurrent neural networks (RNNs). Recent literature exploring RNNs has largely focused on syntactic and semantic knowledge. We extend this domain of investigation to representations of events within RNNs. In three studies, we find preliminary evidence that RNNs capture, in their internal representations, the extent to which objects change states; for example, that chopping an onion changes the onion by more than just peeling the onion. Moreover, the temporal relationship between state changes is encoded to some extent. We found RNNs are sensitive to how chopping an onion and then weighing it, or first weighing it, entails the onion that is being weighed being in a different state depending on the adverb. Our final study explored what factors influence the propagation of these rudimentary event representations forward into subsequent sentences. We conclude that while there is much still to be learned about the abilities of RNNs (especially in respect of the extent to which they encode objects as specific tokens), we still do not know what are the equivalent representational dynamics in humans. That is, we take the perspective that the exploration of computational models points us to important questions about the nature of the human mind.

Representation of linguistic form and function in recurrent neural networks

Residual Recurrent Neural Networks for Learning Sequential Representations.

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Visualizing and Understanding Neural Models in NLP

Colorless green recurrent networks dream hierarchically

Analysis of Argument Structure Constructions in a Deep Recurrent Language Model

Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations

Understanding Hidden Memories of Recurrent Neural Networks

Analyzing Linguistic Knowledge in Sequential Model of Sentence.

Evaluating Recurrent Neural Network Explanations

Recurrent Memory Networks for Language Modeling

Investigating Gated Recurrent Neural Networks for Acoustic Modeling

Recurrently Controlled Recurrent Networks

On Efficiently Representing Regular Languages as RNNs

Finding event structure in time: What recurrent neural networks can tell us about event structure in mind

Lower Bounds on the Expressivity of Recurrent Neural Language Models

Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition

Lyapunov-Guided Representation of Recurrent Neural Network Performance

Geometry of naturalistic object representations in recurrent neural network models of working memory

Learning Useful Representations of Recurrent Neural Network Weight Matrices

Encoding of phonology in a recurrent neural model of grounded speech