Renato Tinós,Michal W. Przewozniczek,Darrell Whitley,Francisco Chicano
Abstract:In pseudo-Boolean optimization, a variable interaction graph represents variables as vertices, and interactions between pairs of variables as edges. In black-box optimization, the variable interaction graph may be at least partially discovered by using empirical linkage learning techniques. These methods never report false variable interactions, but they are computationally expensive. The recently proposed local search with linkage learning discovers the partial variable interaction graph as a side-effect of iterated local search. However, information about the strength of the interactions is not learned by the algorithm. We propose local search with linkage learning 2, which builds a weighted variable interaction graph that stores information about the strength of the interaction between variables. The weighted variable interaction graph can provide new insights about the optimization problem and behavior of optimizers. Experiments with NK landscapes, knapsack problem, and feature selection show that local search with linkage learning 2 is able to efficiently build weighted variable interaction graphs. In particular, experiments with feature selection show that the weighted variable interaction graphs can be used for visualizing the feature interactions in machine learning. Additionally, new transformation operators that exploit the interactions between variables can be designed. We illustrate this ability by proposing a new perturbation operator for iterated local search.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to solve the problem of discovering and exploiting variable interaction structures in pseudo - Boolean optimization. Specifically, the author proposes a new local search strategy - Iterated Local Search with Linkage Learning 2 (LSwLL2) to construct a weighted variable interaction graph (VIGw). VIGw not only represents the interaction relationships between variables but also stores information about the strength of these interactions.
#### Main problems and solutions:
1. **Discovery of variable interaction structures**:
- In black - box optimization, the variable interaction graph (VIG) can be partially discovered through empirical linkage learning techniques. However, existing methods have problems such as high computational cost or inability to report false interactions.
- The method LSwLL2 proposed in this paper can automatically construct VIGw during the process of iterated local search without additional fitness evaluations, thus reducing the computational cost.
2. **Using variable interaction information to improve optimization algorithms**:
- Existing local search strategies (such as LSwLL) can only construct unweighted VIGs and cannot provide information on interaction strength. This is less efficient when dealing with dense VIGs.
- LSwLL2 can better understand the nature of the optimization problem and design more effective transformation operators and search strategies by constructing VIGw. For example, a new perturbation strategy based on VIGw is proposed, which can improve the performance of iterated local search (ILS).
3. **Applications in feature selection and machine learning**:
- Experiments show that VIGw can be used to visualize feature interactions in machine - learning datasets, providing new insights for tasks such as feature selection.
- The new perturbation strategy can accelerate local search in ILS. In particular, when some decision variables have no interaction with other variables, these variables can be independently optimized and fixed, thus speeding up the search.
### Formula summary:
- Fitness change after variable flipping:
\[
\Delta_i(x) = f(x \oplus 1_i) - f(x)
\]
- Difference in fitness change after flipping two variables:
\[
\Delta_{i,j}(x) = |\Delta_i(x \oplus 1_j) - \Delta_i(x)|
\]
- Definition of variable interaction strength:
\[
\omega(i,j) = \frac{1}{2^n} \sum_{x \in B^n} \Delta_{i,j}(x)
\]
- Calculation of edge weights in empirical weighted VIG:
\[
\hat{\omega}(i,j) = \frac{1}{|\Upsilon(i,j)|} \sum_{x \in \Upsilon(i,j)} \Delta_{i,j}(x)
\]
Here, \(B^n\) represents the n - dimensional Boolean space, and \(\Upsilon(i,j)\) is a subset of candidate solutions used to calculate partial sums.
Through these improvements, the paper shows how to more effectively use variable interaction information to improve the performance of optimization algorithms, especially when dealing with complex combinatorial optimization problems.