Abstract:Hopfield neural network (HNN) is a nonlinear computational model successfully applied in finding near-optimal solutions of several difficult combinatorial problems. In many cases, the network energy function is obtained through a learning procedure so that its minima are states falling into a proper subspace (feasible region) of the search space. However, because of the network nonlinearity, a number of undesirable local energy minima emerge from the learning procedure, significantly effecting the network performance. In the neural model analyzed here, we combine both a penalty and a stochastic process in order to enhance the performance of a binary HNN. The penalty strategy allows us to gradually lead the search towards states representing feasible solutions, so avoiding oscillatory behaviors or asymptotically instable convergence. Presence of stochastic dynamics potentially prevents the network to fall into shallow local minima of the energy function, i.e., quite far from global optimum. Hence, for a given fixed network topology, the desired final distribution on the states can be reached by carefully modulating such process. The model uses pseudo-Boolean functions both to express problem constraints and cost function; a combination of these two functions is then interpreted as energy of the neural network. A wide variety of NP-hard problems fall in the class of problems that can be solved by the model at hand, particularly those having a monotonic quadratic pseudo-Boolean function as constraint function. That is, functions easily derived by closed algebraic expressions representing the constraint structure and easy (polynomial time) to maximize. We show the asymptotic convergence properties of this model characterizing its state space distribution at thermal equilibrium in terms of Markov chain and give evidence of its ability to find high quality solutions on benchmarks and randomly generated instances of two specific problems taken from the computational graph theory.

A new mechanical approach to handle generalized Hopfield neural networks

Hebbian Learning from First Principles

On the equivalence of Hopfield networks and Boltzmann Machines

Boltzmann Machines as Generalized Hopfield Networks: A Review of Recent Results and Outlooks

The thermodynamic limit in mean field neural networks

Nonparametric Modern Hopfield Models

Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models

Adaptiveness in monotone pseudo-Boolean optimization and stochastic neural computation

Non-Convex Multi-species Hopfield models

Accelerating Hopfield Network Dynamics: Beyond Synchronous Updates and Forward Euler

A Generalized Hopfield Network for Nonsmooth Constrained Convex Optimization: Lie Derivative Approach

Sparse Hopfield network reconstruction with $\ell_{1}$ regularization

Statistical Mechanics of a Hopfield Neural-Network Model in a Transverse Field

Effect of spatial correlations on Hopfield Neural Network and Dense Associative Memories

Statistical Mechanics of a Multiconnected Hopfield Neural-Network Model in a Transverse Field

Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones

Prototype Analysis in Hopfield Networks with Hebbian Learning

Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models

Input-Driven Dynamics for Robust Memory Retrieval in Hopfield Networks

A generalized Hopfield model to store and retrieve mismatched memory patterns