Distributed Constrained Combinatorial Optimization leveraging Hypergraph Neural Networks

Nasimeh Heydaribeni,Xinrui Zhan,Ruisi Zhang,Tina Eliassi-Rad,Farinaz Koushanfar

2024-05-17

Abstract:Scalable addressing of high dimensional constrained combinatorial optimization problems is a challenge that arises in several science and engineering disciplines. Recent work introduced novel application of graph neural networks for solving quadratic-cost combinatorial optimization problems. However, effective utilization of models such as graph neural networks to address general problems with higher order constraints is an unresolved challenge. This paper presents a framework, HypOp, which advances the state of the art for solving combinatorial optimization problems in several aspects: (i) it generalizes the prior results to higher order constrained problems with arbitrary cost functions by leveraging hypergraph neural networks; (ii) enables scalability to larger problems by introducing a new distributed and parallel training architecture; (iii) demonstrates generalizability across different problem formulations by transferring knowledge within the same hypergraph; (iv) substantially boosts the solution accuracy compared with the prior art by suggesting a fine-tuning step using simulated annealing; (v) shows a remarkable progress on numerous benchmark examples, including hypergraph MaxCut, satisfiability, and resource allocation problems, with notable run time improvements using a combination of fine-tuning and distributed training techniques. We showcase the application of HypOp in scientific discovery by solving a hypergraph MaxCut problem on NDC drug-substance hypergraph. Through extensive experimentation on various optimization problems, HypOp demonstrates superiority over existing unsupervised learning-based solvers and generic optimization methods.

Optimization and Control,Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

This paper aims to address the scalability challenges in high - dimensional constrained combinatorial optimization problems. Specifically, the paper proposes a new framework - HypOp, which significantly improves on existing techniques in the following aspects:

1. **Generality**: HypOp generalizes previous research results to constrained optimization problems with arbitrary cost functions.
2. **Dimensional Expansion**: By adopting a hypergraph neural network structure, HypOp can handle higher - dimensional problems.
3. **Scalability**: A new distributed and parallel architecture for training hypergraph neural networks is introduced, enabling the handling of larger - scale problems.
4. **Transfer Learning**: By transferring the learning experience from one cost/constraint set to another cost/constraint set, HypOp demonstrates its generalization ability in other problem formulations on the same hypergraph.
5. **Accuracy Improvement**: By using simulated annealing for fine - tuning, HypOp significantly improves the accuracy of the solution.
6. **Runtime Optimization**: Combining fine - tuning and distributed training techniques, HypOp improves the runtime by up to 5 times in the benchmark tests.

The HypOp framework allows a series of new scientific problems to be solved, including the hypergraph maximum cut problem, satisfiability problems (such as 3SAT), and resource allocation problems. By solving the hypergraph maximum cut problem on the NDC drug - substance hypergraph, the paper demonstrates the application of HypOp in scientific discovery. Extensive experiments show that HypOp outperforms existing unsupervised learning solvers and general optimization methods on various combinatorial optimization problems.