Abstract:Why do biological and artificial neurons sometimes modularise, each encoding a single meaningful variable, and sometimes entangle their representation of many variables? In this work, we develop a theory of when biologically inspired representations -- those that are nonnegative and energy efficient -- modularise with respect to source variables (sources). We derive necessary and sufficient conditions on a sample of sources that determine whether the neurons in an optimal biologically-inspired linear autoencoder modularise. Our theory applies to any dataset, extending far beyond the case of statistical independence studied in previous work. Rather, we show that sources modularise if their support is "sufficiently spread". From this theory, we extract and validate predictions in a variety of empirical studies on how data distribution affects modularisation in nonlinear feedforward and recurrent neural networks trained on supervised and unsupervised tasks. Furthermore, we apply these ideas to neuroscience data. First, we explain why two studies that recorded prefrontal activity in working memory tasks conflict on whether memories are encoded in orthogonal subspaces: the support of the sources differed due to a critical discrepancy in experimental protocol. Second, we use similar arguments to understand why preparatory and potent subspaces in RNN models of motor cortex are only sometimes orthogonal. Third, we study spatial and reward information mixing in entorhinal recordings, and show our theory matches data better than previous work. And fourth, we suggest a suite of surprising settings in which neurons can be (or appear) mixed selective, without requiring complex nonlinear readouts as in traditional theories. In sum, our theory prescribes precise conditions on when neural activities modularise, providing tools for inducing and elucidating modular representations in brains and machines.

Modular Mechanistic Networks: On Bridging Mechanistic and Phenomenological Models with Deep Neural Networks in Natural Language Processing

Neural language models for network configuration: Opportunities and reality check

Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)

Progress in Neural NLP: Modeling, Learning, and Reasoning

Modelling Compositionality and Structure Dependence in Natural Language

Modularity in Deep Learning: A Survey

Linguistic generalization and compositionality in modern artificial neural networks

Syntactic Structure from Deep Learning

The New Mechanistic Approach and Cognitive Ontology—Or: What Role do (Neural) Mechanisms Play in Cognitive Ontology?

Deep Connectomics Networks: Neural Network Architectures Inspired by Neuronal Networks

Interactive Natural Language Acquisition in a Multi-modal Recurrent Neural Architecture

Modular representations emerge in neural networks trained to perform context-dependent tasks

Towards a neural architecture of language: Deep learning versus logistics of access in neural architectures for compositional processing

Bridging auditory perception and natural language processing with semantically informed deep neural networks

Deep Learning and Machine Learning -- Natural Language Processing: From Theory to Application

Breaking Neural Network Scaling Laws with Modularity

An Alternative to Cognitivism: Computational Phenomenology for Deep Learning

Don't Cut Corners: Exact Conditions for Modularity in Biologically Inspired Representations

Understanding the Dynamics of DNNs Using Graph Modularity

Topological properties and organizing principles of semantic networks

Mechanistic Neural Networks for Scientific Machine Learning