Abstract:Striking progress has been made in understanding cognition by analyzing how the brain is engaged in different modes of information processing. For instance, so-called synergistic information (information encoded by a set of neurons but not by any subset) plays a key role in areas of the human brain linked with complex cognition. However, two questions remain unanswered: (a) how and why a cognitive system can become highly synergistic; and (b) how informational states map onto artificial neural networks in various learning modes. Here we employ an information-decomposition framework to investigate neural networks performing cognitive tasks. Our results show that synergy increases as networks learn multiple diverse tasks, and that in tasks requiring integration of multiple sources, performance critically relies on synergistic neurons. Overall, our results suggest that synergy is used to combine information from multiple modalities—and more generally for flexible and efficient learning. These findings reveal new ways of investigating how and why learning systems employ specific information-processing strategies, and support the principle that the capacity for general-purpose learning critically relies on the system's information dynamics. What is the informational basis of learning in humans, animals, or, indeed, artificial neural networks (ANN)? Furthermore, how can these systems learn to solve multiple tasks simultaneously? These fundamental questions are, surprisingly, still not fully understood. One advantage of studying ANNs is that we can precisely probe learning-related changes. Here we draw on a recent branch of information theory, partial information decomposition, to examine how different types of information support different learning goals, and where, in ANNs. We show that adding noise to an ANN encourages it to keep copies of information at multiple nodes, promoting robustness. In contrast, whenever flexible learning is required, for instance when facing varied stimulus types or diverse tasks, individual neurons work together to represent information more abstractly. This work sheds light on how systems encode information differently according to their learning pressures, which can help us better understand how and why the human brain uses particular forms of information processing.

Synergistic pathways of modulation enable robust task packing within neural dynamics

Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks

Neuromodulators generate multiple context-relevant behaviors in a recurrent neural network by shifting activity flows in hyperchannels

Structured flexibility in recurrent neural networks via neuromodulation

Emergence and reconfiguration of modular structure for synaptic neural networks during continual familiarity detection

Neural Computations in A Dynamical System with Multiple Time Scales

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Modelling continual learning in humans with Hebbian context gating and exponentially decaying task signals

Task representations in neural networks trained to perform many cognitive tasks

Building compositional tasks with shared neural subspaces

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

A Unified Computational Model of the Genetic Regulatory Networks Underlying Synaptic, Intrinsic and Homeostatic Plasticity

The brain as an efficient and robust adaptive learner

Engineering recurrent neural networks from task-relevant manifolds and dynamics

Evolving Generalized Modulatory Learning: Unifying Neuromodulation and Synaptic Plasticity

The Collective Bursting Dynamics in A Modular Neuronal Network with Synaptic Plasticity

Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling

Structure of activity in multiregion recurrent neural networks

Induction and Propagation of Transient Synchronous Activity in Neural Networks Endowed with Short-Term Plasticity

Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimes

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation