ReLU-type Hopfield Neural Network with Analog Hardware Implementation

Chengjie Chen,Fuhong Min,Yunzhen Zhang,Han Bao
DOI: https://doi.org/10.1016/j.chaos.2022.113068
2022-01-01
Abstract:Rectified Linear Unit (ReLU) is an extremely simple activation function with two-segment linearity and is usually used in convolutional neural network. To simplify the analog hardware implementation, this article presents a bi-neuron Hopfield neural network (HNN) using ReLU function instead of hyperbolic tangent (Tanh) function as the activation function. With the HNN model, the boundedness properties can be proved by virtue of the Lyapunov method and the stability distributions of time-varying equilibrium point are clarified according to four independent affine regions. Using numerical measures, complex dynamical behaviors are uncovered therein, including period-adding bifurcation, spiking/bursting pattern, fast-slow effect, and coexisting multistable patterns. In addition, a multiplierless electronic circuit is developed for implementing the HNN model and its experimental measurements well validate the numerical ones. The results manifest that the ReLU-type HNN model can display intricate and rich dynamics, and its analog hardware implementation is simpler than other HNN models.
What problem does this paper attempt to address?