Abstract:Neurons communicate nonlinearly through spike activities. Generalized linear models (GLMs) describe spike activities with a cascade of a linear combination across inputs, a static nonlinear function, and an inhomogeneous Bernoulli or Poisson process, or Cox process if a self-history term is considered. This structure considers the output nonlinearity in spike generation but excludes the nonlinear interaction among input neurons. Recent studies extend GLMs by modeling the interaction among input neurons with a quadratic function, which considers the interaction between every pair of input spikes. However, quadratic effects may not fully capture the nonlinear nature of input interaction. We therefore propose a staged point-process model to describe the nonlinear interaction among inputs using a few hidden units, which follows the idea of artificial neural networks. The output firing probability conditioned on inputs is formed as a cascade of two linear-nonlinear (a linear combination plus a static nonlinear function) stages and an inhomogeneous Bernoulli process. Parameters of this model are estimated by maximizing the log likelihood on output spike trains. Unlike the iterative reweighted least squares algorithm used in GLMs, where the performance is guaranteed by the concave condition, we propose a modified Levenberg-Marquardt(L-M) algorithm, which directly calculates the Hessian matrix of the log likelihood, for the nonlinear optimization in our model. The proposed model is tested on both synthetic data and real spike train data recorded from the dorsal premotor cortex and primary motor cortex of a monkey performing a center-out task. Performances are evaluated by discrete-time rescaled Kolmogorov-Smirnov tests, where our model statistically outperforms a GLM and its quadratic extension, with a higher goodness-of-fit in the prediction results. In addition, the staged point-process model describes nonlinear interaction among input neurons with fewer parameters than quadratic models, and the modified L-M algorithm also demonstrates fast convergence.

A Two-stage Sparse Optimization Algorithm for Gated Recurrent Units Estimators

Modeling a Delayed Coking Process with GRNN and Double-Chain Based DNA Genetic Algorithm

An Adaptive Variable Selection Algorithm for Gated Recurrent Unit Based on Sensitivity Analysis and Nonnegative Garrote

An Accurate GRU-Based Power Time-Series Prediction Approach With Selective State Updating and Stochastic Optimization

Nonlinear Modeling of Neural Interaction for Spike Prediction Using the Staged Point-Process Model

Causal Variable Selection for Industrial Process Quality Prediction Via Attention-Based GRU Network

Multi-grain Cascade Recurrent Neural Network for Nonlinear Time-varying Process Soft Sensor Modeling

A Fast Accurate Two-stage Training Algorithm for L1-regularized CRFs with Heuristic Line Search Strategy.

Input Variable Selection and Structure Optimization for LSTM-Based Soft Sensor With a Dual Nonnegative Garrote Approach

From Bayesian Sparsity to Gated Recurrent Nets

Genetic Programming Based On An Adaptive Regularization Method

Predicting Grain Losses and Waste Rate Along the Entire Chain: A Multitask Multigated Recurrent Unit Autoencoder Based Method

A Parallel Gated Recurrent Units (P-GRUs) network for the shifting lateness bottleneck prediction in make-to-order production system

Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units

Development of input variable selection and structural optimization algorithm for recurrent neural network

Deep transition network with gating mechanism for multivariate time series forecasting

Wireless Channel Prediction of GRU Based on Experience Replay and Snake Optimizer

A Hybrid Variable Selection Algorithm for Multi-Layer Perceptron with Nonnegative Garrote and Extremal Optimization

NGCU: A New RNN Model for Time-Series Data Prediction

Gating creates slow modes and controls phase-space complexity in GRUs and LSTMs

Sequence-to-Sequence Stacked Gate Recurrent Unit Networks for Approximating the Forward Problem of Partial Differential Equations