Abstract:Striking progress has been made in understanding cognition by analyzing how the brain is engaged in different modes of information processing. For instance, so-called synergistic information (information encoded by a set of neurons but not by any subset) plays a key role in areas of the human brain linked with complex cognition. However, two questions remain unanswered: (a) how and why a cognitive system can become highly synergistic; and (b) how informational states map onto artificial neural networks in various learning modes. Here we employ an information-decomposition framework to investigate neural networks performing cognitive tasks. Our results show that synergy increases as networks learn multiple diverse tasks, and that in tasks requiring integration of multiple sources, performance critically relies on synergistic neurons. Overall, our results suggest that synergy is used to combine information from multiple modalities—and more generally for flexible and efficient learning. These findings reveal new ways of investigating how and why learning systems employ specific information-processing strategies, and support the principle that the capacity for general-purpose learning critically relies on the system's information dynamics. What is the informational basis of learning in humans, animals, or, indeed, artificial neural networks (ANN)? Furthermore, how can these systems learn to solve multiple tasks simultaneously? These fundamental questions are, surprisingly, still not fully understood. One advantage of studying ANNs is that we can precisely probe learning-related changes. Here we draw on a recent branch of information theory, partial information decomposition, to examine how different types of information support different learning goals, and where, in ANNs. We show that adding noise to an ANN encourages it to keep copies of information at multiple nodes, promoting robustness. In contrast, whenever flexible learning is required, for instance when facing varied stimulus types or diverse tasks, individual neurons work together to represent information more abstractly. This work sheds light on how systems encode information differently according to their learning pressures, which can help us better understand how and why the human brain uses particular forms of information processing.

A Review on Synergistic Learning.

Synergies Between Intrinsic and Synaptic Plasticity Based on Information Theoretic Learning

Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks

The next generation of approaches to investigate the link between synaptic plasticity and learning

Hybrid Dual-Complementary Metal-Oxide-semiconductor/memristor Synapse-Based Neural Network with Its Applications in Image Super-Resolution

Brain-inspired learning in artificial neural networks: a review

Evolving Generalized Modulatory Learning: Unifying Neuromodulation and Synaptic Plasticity

Control of synaptic plasticity via the fusion of reinforcement learning and unsupervised learning in neural networks

Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks

Computational models of learning and synaptic plasticity

Brain-inspired Global-Local Learning Incorporated with Neuromorphic Computing

Synergetic Learning Systems: Concept, Architecture, and Algorithms

Co-dependent excitatory and inhibitory plasticity accounts for quick, stable and long-lasting memories in biological networks

Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity

[Progress on Metaplasticity and Its Role in Learning and Memory].

Synergistic pathways of modulation enable robust task packing within neural dynamics

Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks

Recent Advance in Synaptic Plasticity Modulation Techniques for Neuromorphic Applications

The Demise of the Synapse As the Locus of Memory: A Looming Paradigm Shift?