Abstract:Recent success in training deep neural networks have prompted active investigation into the features learned on their intermediate layers. Such research is difficult because it requires making sense of non-linear computations performed by millions of parameters, but valuable because it increases our ability to understand current models and create improved versions of them. In this paper we investigate the extent to which neural networks exhibit what we call convergent learning, which is when the representations learned by multiple nets converge to a set of features which are either individually similar between networks or where subsets of features span similar low-dimensional spaces. We propose a specific method of probing representations: training multiple networks and then comparing and contrasting their individual, learned representations at the level of neurons or groups of neurons. We begin research into this question using three techniques to approximately align different neural networks on a feature level: a bipartite matching approach that makes one-to-one assignments between neurons, a sparse prediction approach that finds one-to-many mappings, and a spectral clustering approach that finds many-to-many mappings. This initial investigation reveals a few previously unknown properties of neural networks, and we argue that future research into the question of convergent learning will yield many more. The insights described here include (1) that some features are learned reliably in multiple networks, yet other features are not consistently learned; (2) that units learn to span low-dimensional subspaces and, while these subspaces are common to multiple networks, the specific basis vectors learned are not; (3) that the representation codes show evidence of being a mix between a local code and slightly, but not fully, distributed codes across multiple units.

Convergence Theorems for the Kohonen Feature Mapping Algorithms with VLRPs

Convergence Theorems for a Class of Learning Algorithms with VLRPs

Convergence Analysis of General Neural Networks under Almost Periodic Stimuli

Convergence Analysis of Graph Regularized Non-Negative Matrix Factorization

Convergence analysis of discrete time recurrent neural networks for linear variational inequality problem

Convergence Rates of Training Deep Neural Networks Via Alternating Minimization Methods.

A Novel Global Training Algorithm and Its Convergence Theorem for Fuzzy Neural Networks

A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

Fast and Strong Convergence of Online Learning Algorithms.

Convergence Analysis of Two-layer Neural Networks with ReLU Activation

On Convergence of Training Loss Without Reaching Stationary Points

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization

Convergence of Some Convex Message Passing Algorithms to a Fixed Point

Convergent Learning: Do different neural networks learn the same representations?

Stochastic three-term conjugate gradient method with variance technique for non-convex learning

A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network

On the Convergence Analysis of Over-Parameterized Variational Autoencoders: A Neural Tangent Kernel Perspective

Learning-rate-free Momentum SGD with Reshuffling Converges in Nonsmooth Nonconvex Optimization

On the Convergence of Gradient Descent for Large Learning Rates

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimiax Optimization

The Correct Convergence of the Rival Penalized Competitive Learning (rpcl) Algorithm