Abstract:Striking progress has been made in understanding cognition by analyzing how the brain is engaged in different modes of information processing. For instance, so-called synergistic information (information encoded by a set of neurons but not by any subset) plays a key role in areas of the human brain linked with complex cognition. However, two questions remain unanswered: (a) how and why a cognitive system can become highly synergistic; and (b) how informational states map onto artificial neural networks in various learning modes. Here we employ an information-decomposition framework to investigate neural networks performing cognitive tasks. Our results show that synergy increases as networks learn multiple diverse tasks, and that in tasks requiring integration of multiple sources, performance critically relies on synergistic neurons. Overall, our results suggest that synergy is used to combine information from multiple modalities—and more generally for flexible and efficient learning. These findings reveal new ways of investigating how and why learning systems employ specific information-processing strategies, and support the principle that the capacity for general-purpose learning critically relies on the system's information dynamics. What is the informational basis of learning in humans, animals, or, indeed, artificial neural networks (ANN)? Furthermore, how can these systems learn to solve multiple tasks simultaneously? These fundamental questions are, surprisingly, still not fully understood. One advantage of studying ANNs is that we can precisely probe learning-related changes. Here we draw on a recent branch of information theory, partial information decomposition, to examine how different types of information support different learning goals, and where, in ANNs. We show that adding noise to an ANN encourages it to keep copies of information at multiple nodes, promoting robustness. In contrast, whenever flexible learning is required, for instance when facing varied stimulus types or diverse tasks, individual neurons work together to represent information more abstractly. This work sheds light on how systems encode information differently according to their learning pressures, which can help us better understand how and why the human brain uses particular forms of information processing.

Adaptive learning is structure learning in time

Flexible structure learning under uncertainty

Asymmetric learning and adaptability to changes in relational structure during transitive inference

Built to Adapt: Mechanisms of Cognitive Flexibility in the Human Brain

The Adaptive Learning System Based on Learning Style and Cognitive State

Exploiting Cognitive Structure for Adaptive Learning

Learning at Variable Attentional Load Requires Cooperation of Working Memory, Meta-learning, and Attention-augmented Reinforcement Learning

Learning and Cognitive Flexibility: Frontostriatal Function and Monoaminergic Modulation

Capturing Optimal and Suboptimal behavior Of Agents Via Structure Learning Of Their Internal Model

Learning Structure of Complex Sensory Inputs with Plasticity in Neural Microcircuits

Integration of new information in memory: new insights from a complementary learning systems perspective

An algorithmic account for how humans efficiently learn, transfer, and compose hierarchically structured decision policies

Context selectivity with dynamic availability enables lifelong continual learning

Structure and influence in an interconnected world: neurocomputational mechanism of real-time distributed learning on social networks

Incorporating neuro-inspired adaptability for continual learning in artificial intelligence

Rapid learning of temporal dependencies at multiple timescales

Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks

Two determinants of dynamic adaptive learning for magnitudes and probabilities

A neural mechanism for compositional generalization of structure in humans

Towards learning-to-learn

Learning the structure of the world: The adaptive nature of state-space and action representations in multi-stage decision-making