Abstract:Advances in optical and electrophysiological recording technologies have made it possible to record the dynamics of thousands of neurons, opening up new possibilities for interpreting and controlling large neural populations in behaving animals. A promising way to extract computational principles from these large datasets is to train data-constrained recurrent neural networks (dRNNs). Performing this training in real-time could open doors for research techniques and medical applications to model and control interventions at single-cell resolution and drive desired forms of animal behavior. However, existing training algorithms for dRNNs are inefficient and have limited scalability, making it a challenge to analyze large neural recordings even in offline scenarios. To address these issues, we introduce a training method termed Convex Optimization of Recurrent Neural Networks (CORNN). In studies of simulated recordings, CORNN attained training speeds ~100-fold faster than traditional optimization approaches while maintaining or enhancing modeling accuracy. We further validated CORNN on simulations with thousands of cells that performed simple computations such as those of a 3-bit flip-flop or the execution of a timed response. Finally, we showed that CORNN can robustly reproduce network dynamics and underlying attractor structures despite mismatches between generator and inference models, severe subsampling of observed neurons, or mismatches in neural time-scales. Overall, by training dRNNs with millions of parameters in subminute processing times on a standard computer, CORNN constitutes a first step towards real-time network reproduction constrained on large-scale neural recordings and a powerful computational tool for advancing the understanding of neural computation.

Learning Korobov Functions by Correntropy and Convolutional Neural Networks

A Regularized Correntropy Framework for Robust Pattern Recognition

Advances in Convolutional Neural Networks

Generalized Correntropy Induced Loss Function for Deep Learning.

Robust Regression Via Deep Negative Correlation Learning

Learning with the Maximum Correntropy Criterion Induced Losses for Regression.

Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Norms

Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations

Learning with convolution and pooling operations in kernel methods

Generalizing the Convolution Operator in Convolutional Neural Networks

CORNN: Convex optimization of recurrent neural networks for rapid inference of neural dynamics

Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks

On the Learning Dynamics of Two-layer Nonlinear Convolutional Neural Networks.

Correntropy-Induced Robust Low-Rank Hypergraph.

Efficient and Robust Deep Learning with Correntropy-induced Loss Function

Correlative Filters For Convolutional Neural Networks

Markonv: a Novel Convolutional Layer with Inter-Positional Correlations Modeled

CovNN: A Covariance Neural Network Extended from CNN

Multi-Kernel Correntropy for Robust Learning

Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks

Correlation-based Structural Dropout for Convolutional Neural Networks