Search Strategy Generation for Branch and Bound Using Genetic Programming

Gwen Maudet,Grégoire Danoy

2024-12-17

Abstract:Branch-and-Bound (B\&B) is an exact method in integer programming that recursively divides the search space into a tree. During the resolution process, determining the next subproblem to explore within the tree-known as the search strategy-is crucial. Hand-crafted heuristics are commonly used, but none are effective over all problem classes. Recent approaches utilizing neural networks claim to make more intelligent decisions but are computationally expensive. In this paper, we introduce GP2S (Genetic Programming for Search Strategy), a novel machine learning approach that automatically generates a B\&B search strategy heuristic, aiming to make intelligent decisions while being computationally lightweight. We define a policy as a function that evaluates the quality of a B\&B node by combining features from the node and the problem; the search strategy policy is then defined by a best-first search based on this node ranking. The policy space is explored using a genetic programming algorithm, and the policy that achieves the best performance on a training set is selected. We compare our approach with the standard method of the SCIP solver, a recent graph neural network-based method, and handcrafted heuristics. Our first evaluation includes three types of primal hard problems, tested on instances similar to the training set and on larger instances. Our method is at most 2\% slower than the best baseline and consistently outperforms SCIP, achieving an average speedup of 11.3\%. Additionally, GP2S is tested on the MIPLIB 2017 dataset, generating multiple heuristics from different subsets of instances. It exceeds SCIP's average performance in 7 out of 10 cases across 15 times more instances and under a time limit 15 times longer, with some GP2S methods leading on most experiments in terms of the number of feasible solutions or optimality gap.

Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to automatically generate an efficient and computationally less - costly search strategy (SS) for the Branch - and - Bound (B&B) method to improve the efficiency of solving integer programming problems. Specifically, the paper aims to overcome the limitations of existing methods:

1. **Manually - designed heuristic algorithms**: These algorithms perform inconsistently on different types of instances and lack universal applicability.
2. **Neural - network - based methods**: Although they can make more intelligent decisions, they have a high computational cost and are difficult to be widely applied.

To this end, the paper proposes GP2S (Genetic Programming for Search Strategy), a new method that uses genetic programming to automatically generate B&B search strategies. This method defines a scoring function to evaluate the quality of B&B nodes and uses the Best - First Search (BFS) to select the next node to be explored based on this scoring function. It explores the strategy space through genetic programming to find the strategy with the best performance on the training set.

### The main contributions of the paper include:
1. **Defining the search strategy space**: Define a scoring function based on nodes and problem data, covering existing manually - designed heuristic algorithms.
2. **Designing a genetic programming algorithm**: Used to explore this strategy space, aiming to improve the performance of B&B on the instance set.
3. **Experimental results show**: GP2S performs well on a variety of test sets, especially on larger - scale instances, and its performance is better than benchmark methods such as SCIP, with an average speedup of 11.3%.
4. **Evaluation of the MIPLIB 2017 data set**: Verifies the generalization ability of GP2S on heterogeneous data sets and its efficiency within the time limit.

### The importance of search strategies
In the B&B algorithm, the search strategy determines the order of sub - problems in the exploration tree, which has a significant impact on the speed and efficiency of finding the optimal solution. Therefore, a good search strategy can significantly reduce the solution time and improve the pruning efficiency.

### Formula representation
To ensure the correctness and readability of the formulas, the following are some of the formulas involved in the paper represented in Markdown format:

- **Mathematical representation of the MILP problem**:
\[
\min_{x \in N_0} \{c^T x : A x \geq b\}
\]
where \(N_0=\mathbb{Z}^k\times\mathbb{R}^{n - k}\), \(A\in\mathbb{R}^{m\times n}\), \(b\in\mathbb{R}^m\), \(c\in\mathbb{R}^n\).

- **Example of node scoring function**:
\[
f(N_i)=z_i - M\times d_i
\]
where \(z_i\) is the lower bound of node \(i\), \(d_i\) is the depth of node \(i\), and \(M\) is a sufficiently large positive constant.

Through these improvements, GP2S can provide an efficient search strategy while ensuring computational lightness, thereby improving the efficiency of solving integer programming problems.