Abstract:Striking progress has been made in understanding cognition by analyzing how the brain is engaged in different modes of information processing. For instance, so-called synergistic information (information encoded by a set of neurons but not by any subset) plays a key role in areas of the human brain linked with complex cognition. However, two questions remain unanswered: (a) how and why a cognitive system can become highly synergistic; and (b) how informational states map onto artificial neural networks in various learning modes. Here we employ an information-decomposition framework to investigate neural networks performing cognitive tasks. Our results show that synergy increases as networks learn multiple diverse tasks, and that in tasks requiring integration of multiple sources, performance critically relies on synergistic neurons. Overall, our results suggest that synergy is used to combine information from multiple modalities—and more generally for flexible and efficient learning. These findings reveal new ways of investigating how and why learning systems employ specific information-processing strategies, and support the principle that the capacity for general-purpose learning critically relies on the system's information dynamics. What is the informational basis of learning in humans, animals, or, indeed, artificial neural networks (ANN)? Furthermore, how can these systems learn to solve multiple tasks simultaneously? These fundamental questions are, surprisingly, still not fully understood. One advantage of studying ANNs is that we can precisely probe learning-related changes. Here we draw on a recent branch of information theory, partial information decomposition, to examine how different types of information support different learning goals, and where, in ANNs. We show that adding noise to an ANN encourages it to keep copies of information at multiple nodes, promoting robustness. In contrast, whenever flexible learning is required, for instance when facing varied stimulus types or diverse tasks, individual neurons work together to represent information more abstractly. This work sheds light on how systems encode information differently according to their learning pressures, which can help us better understand how and why the human brain uses particular forms of information processing.

A neural network model of differentiation and integration of competing memories

Differentiation and Integration of Competing Memories: A Neural Network Model

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

Retrospective memory integration accompanies reconfiguration of neural cell assemblies

A model for integrating elementary neural functions into delayed-response behavior

Intrinsic neural excitability biases allocation and overlap of memory engrams

Spiking Neuron Based Cognitive Memory Model

Integration of new information in memory: new insights from a complementary learning systems perspective

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

A Model of the Interaction Between Mood and Memory

Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks

A dynamic attractor network model of memory formation, reinforcement and forgetting

Segregation-to-Integration Transformation Model of Memory Evolution

A Neural Dynamic Model based on Activation Diffusion and a Micro-Explanation for Cognitive Operations

Building and Breaking the Chain: A Model of Reward Prediction Error Integration and Segmentation of Memory

Neural networks simulating short-term memory of two inputs with varying commonality

0058 Neural Differentiation of Competing Memory Representations After a Prediction Error Depends on REM Sleep

A simple neural circuit model explains diverse types of integration kernels in perceptual decision-making

On the Nature of Functional Differentiation: The Role of Self-Organization with Constraints

Kernel Memory Networks: A Unifying Framework for Memory Modeling

Memristive Circuit Design of Associative Memory with Generalization and Differentiation