Abstract:Videogames provide a promising framework to understand brain activity in a rich, engaging, and active environment, in contrast to mostly passive tasks currently dominating the field, such as image viewing. Analyzing videogames neuroimaging data is however challenging, and relies on time-intensive manual annotations of game events, based on somewhat arbitrary rules. Here, we introduce an innovative approach using Artificial Neural networks (ANN) and brain encoding techniques to generate activation maps associated with videogame behaviour using functional magnetic resonance imaging (fMRI). As individual behavior is highly variable across subjects in complex environments, we hypothesized that ANNs need to account for subject-specific behavior to properly capture brain dynamics. In this study, we used data collected while subjects played Shinobi III: Return of the Ninja Master (Sega, 1993), an action-platformer videogame. Using imitation learning, we trained an ANN to play the game while closely replicating the unique gameplay style of individual participants. We found that hidden layers of our imitation learning model successfully encoded task-relevant neural representations, and predicted individual brain dynamics with higher accuracy than models trained on other subjects' gameplay. Individual-specific models also outperformed a number of baselines to predict brain activity, such as pixel inputs, or button presses. The highest correlations between layer activations and brain signals were observed in biologically plausible brain areas, i.e. somatosensory, attention, and visual networks. Our results demonstrate that training subject-specific ANNs can successfully uncover brain correlates of complex behaviour. This new method combining imitation learning, brain imaging, and videogames opens new research avenues to study decision-making and psychomotor task solving in naturalistic and complex environments.

Using deep reinforcement learning to reveal how the brain encodes abstract state-space representations in high-dimensional environments

Human-Level Control Through Directly-Trained Deep Spiking Q-Networks

State of the Art Control of Atari Games Using Shallow Reinforcement Learning

Brain Inspired Episodic Memory Deep Q-Networks for Sparse Reward

Behavioral Imitation with Artificial Neural Networks Leads to Personalized Models of Brain Dynamics During Videogame Play

Architecting and Visualizing Deep Reinforcement Learning Models

Deep Reinforcement Learning and its Neuroscientific Implications

Capturing the Complexity of Human Strategic Decision-Making with Machine Learning

Investigating the properties of neural network representations in reinforcement learning

DQN with model-based exploration: efficient learning on environments with sparse rewards

Towards Brain-inspired System: Deep Recurrent Reinforcement Learning for Simulated Self-driving Agent

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Toward a Brain-Inspired System: Deep Recurrent Reinforcement Learning for a Simulated Self-Driving Agent

Curiosity-Driven Variational Autoencoder for Deep Q Network

Forming cognitive maps for abstract spaces: the roles of the human hippocampus and orbitofrontal cortex

Reinforcement Learning and its Connections with Neuroscience and Psychology

Playing Atari with Deep Reinforcement Learning

How Do You Act? An Empirical Study to Understand Behavior of Deep Reinforcement Learning Agents

Deep Reinforcement Learning Models Predict Visual Responses in the Brain: A Preliminary Result

Handling Large-Scale Action Space In Deep Q Network

Hippocampal representations emerge when training recurrent neural networks on a memory dependent maze navigation task