The Gâteaux-Hopfield Neural Network method

Felipe Silva Carvalho,João Pedro Braga

DOI: https://doi.org/10.48550/arXiv.2001.11853

2020-01-30

Abstract:In the present work a new set of differential equations for the Hopfield Neural Network (HNN) method were established by means of the Linear Extended Gateaux Derivative (LEGD). This new approach will be referred to as Gâteaux-Hopfiel Neural Network (GHNN). A first order Fredholm integral problem was used to test this new method and it was found to converge 22 times faster to the exact solutions for {\alpha} > 1 if compared with the HNN integer order differential equations. Also a limit to the learning time is observed by analysing the results for different values of {\alpha}. The robustness and advantages of this new method will be pointed out.

Machine Learning,Neural and Evolutionary Computing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: to improve the Hopfield neural network (HNN) method by introducing the linearly extended Gateaux derivative (LEGD) in order to solve inverse problems more effectively. Specifically, the author proposes a new system of differential equations - the Gateaux - Hopfield neural network (GHNN), and uses it to solve the first - order Fredholm integral equation problem. Compared with the traditional HNN method, the new method has a significant reduction in computing time, especially in the case of the parameter \(\alpha > 1\).

### Main problem description:

1. **Solution of inverse problems**:
- Inverse problems are a class of scientific problems that extract physical and chemical information from experimental data. Such problems usually have an ill - posed nature and require special numerical techniques for regularization and solution.
- For example, recovering molecular force fields from experimental frequencies, recovering probability density functions from experimental positron annihilation lifetime spectra, etc.

2. **Limitations of existing methods**:
- Although the Hopfield neural network (HNN) shows strong robustness in solving inverse problems, its long learning time affects its efficiency.
- Introducing a more general derivative concept can optimize this process.

3. **New method of introducing Gateaux derivative**:
- The paper proposes to use the linearly extended Gateaux derivative (LEGD) to define a new system of differential equations, thus forming the Gateaux - Hopfield neural network (GHNN).
- In this way, the energy dissipation problem can be naturally handled and the basic neural network equations can be simplified.

4. **Testing and verification**:
- The first - order Fredholm integral equation is used as a test case to verify the effectiveness of the GHNN method.
- The results show that for the case of \(\alpha > 1\), the GHNN method converges 22 times faster than the traditional HNN method.

### Mathematical background and formulas:

1. **Linearly extended Gateaux derivative (LEGD)**:
\[
df_{\text{leg}}(t; \psi)=\lim_{\epsilon \to 0}\frac{f(t + \epsilon\psi(t, \alpha)) - f(t)}{\epsilon}
\]

2. **GHNN equation**:
\[
\frac{du}{dt}=\frac{1}{\psi}[-K^{T}Kf + K^{T}g]
\]
where \(\psi = t^{1-\alpha}\), \(\alpha\in\mathbb{R}\).

3. **Generalization of the Euler method**:
\[
u_{i + 1}=u_i+h\left[\frac{1}{\psi(t_i; \alpha)}[K'Kf_i - K'g]\right]
\]

### Summary:

The main objective of this paper is to improve the HNN method by introducing LEGD in order to solve inverse problems more quickly. The GHNN method is not only more theoretically rigorous but also shows a significant improvement in computational efficiency in practical applications.