Better Understandings and Configurations in MaxSAT Local Search Solvers Via Anytime Performance Analysis

Furong Ye,Chuan Luo,Shaowei Cai
DOI: https://doi.org/10.48550/arxiv.2403.06568
2024-01-01
Abstract:Though numerous solvers have been proposed for the MaxSAT problem, and thebenchmark environment such as MaxSAT Evaluations provides a platform for thecomparison of the state-of-the-art solvers, existing assessments were usuallyevaluated based on the quality, e.g., fitness, of the best-found solutionsobtained within a given running time budget. However, concerning solely thefinal obtained solutions regarding specific time budgets may restrict us fromcomprehending the behavior of the solvers along the convergence process. Thispaper demonstrates that Empirical Cumulative Distribution Functions can be usedto compare MaxSAT local search solvers' anytime performance across multipleproblem instances and various time budgets. The assessment reveals distinctionsin solvers' performance and displays that the (dis)advantages of solvers adjustalong different running times. This work also exhibits that the quantitativeand high variance assessment of anytime performance can guide machines, i.e.,automatic configurators, to search for better parameter settings. Ourexperimental results show that the hyperparameter optimization tool, i.e.,SMAC, generally achieves better parameter settings of local search when usingthe anytime performance as the cost function, compared to using the fitness ofthe best-found solutions.
What problem does this paper attempt to address?