Abstract:The physics of the environment provide a rich spatiotemporal structure for our experience. Objects move in predictable ways and their features and identity remain stable across time and space. How does the brain leverage this structure to make predictions about and learn from the environment? This thesis describes research centered around a mechanistic description of sensory prediction called LeabraTI (TI: Temporal Integration) that explains precisely how predictive processing is accomplished in neocortical microcircuits. The fundamental prediction of LeabraTI is that predictions and sensations are interleaved across the same neural tissue at an overall rate of 10 Hz, corresponding to the widely studied alpha rhythm of posterior cortex. Experiments described herein tested this prediction by manipulating the spatiotemporal properties of three-dimensional object stimuli in a laboratory setting. EEG results indicated that predictions were subserved by ~10 Hz oscillations that reliably tracked the onset of stimuli and differentiated between spatially predictable and unpredictable object sequences. There was a behavioral advantage for combined spatial and temporal predictability for discrimination of unlearned objects, but prolonged study of objects under this combined predictability context impaired discriminability relative to other learning contexts. This counterintuitive pattern of results was accounted for by a neural network model that learned three-dimensional viewpoint invariance with LeabraTI's spatiotemporal prediction rule. Synaptic weight scaling from prolonged learning built viewpoint invariance, but led to confusion between ambiguous views of objects, producing slightly lower performance on average. Overall, this work advances a biological architecture for sensory prediction accompanied by empirical evidence that supports learning of realistic time- and space-varying inputs.

Predictive event segmentation and representation with neural networks: A self-supervised model assessed by psychological experiments

Modeling human activity comprehension at human scale: prediction, segmentation, and categorization

The Neural Basis of Event Segmentation: Stable Features in the Environment are Reflected by Neural States

Prediction Error and Event Segmentation in Episodic Memory

Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Neural representations of predicted events: Evidence from time-resolved EEG decoding

Neural Conditional Event Time Models

Boundaries in the eyes: measure event segmentation during naturalistic video watching using eye tracking

Learning, planning, and control in a monolithic neural event inference architecture

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Probing neural representations of scene perception in a hippocampally dependent task using artificial neural networks

An Empirical Evaluation of Neural and Neuro-symbolic Approaches to Real-time Multimodal Complex Event Detection

Unraveling the Neural Network: Identifying Temporal Labeling of Visual Events through EEG-Based Functional Connectivity Analysis of Brain Regions

EvSegSNN: Neuromorphic Semantic Segmentation for Event Data

Latent Event-Predictive Encodings through Counterfactual Regularization

Exploring the Chameleon Effect of Contextual Dynamics in Temporal Knowledge Graph for Event Prediction

What happens next and when "next" happens: Mechanisms of spatial and temporal prediction

Empirically Identifying and Computationally Modeling the Brain-Behavior Relationship for Human Scene Categorization

A Temporal Attention-based Model for Social Event Prediction

Predictive Encoding of Contextual Relationships for Perceptual Inference, Interpolation and Prediction.