Abstract:Heuristics are commonly used to tackle diverse search and optimization problems. Design heuristics usually require tedious manual crafting with domain knowledge. Recent works have incorporated large language models (LLMs) into automatic heuristic search leveraging their powerful language and coding capacity. However, existing research focuses on the optimal performance on the target problem as the sole objective, neglecting other criteria such as efficiency and scalability, which are vital in practice. To tackle this challenge, we propose to model heuristic search as a multi-objective optimization problem and consider introducing other practical criteria beyond optimal performance. Due to the complexity of the search space, conventional multi-objective optimization methods struggle to effectively handle multi-objective heuristic search. We propose the first LLM-based multi-objective heuristic search framework, Multi-objective Evolution of Heuristic (MEoH), which integrates LLMs in a zero-shot manner to generate a non-dominated set of heuristics to meet multiple design criteria. We design a new dominance-dissimilarity mechanism for effective population management and selection, which incorporates both code dissimilarity in the search space and dominance in the objective space. MEoH is demonstrated in two well-known combinatorial optimization problems: the online Bin Packing Problem (BPP) and the Traveling Salesman Problem (TSP). Results indicate that a variety of elite heuristics are automatically generated in a single run, offering more trade-off options than existing methods. It successfully achieves competitive or superior performance while improving efficiency up to 10 times. Moreover, we also observe that the multi-objective search introduces novel insights into heuristic design and leads to the discovery of diverse heuristics.

A meta-heuristic extension of the Lagrangian heuristic framework

An enhanced heuristic searching algorithm for complicated constrained optimization problems

In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics

A probabilistic framework with the gradient-based method for multi-objective land use optimization

Foundations of the Pareto Iterated Local Search Metaheuristic

Multi-objective Evolution of Heuristic Using Large Language Model

Landscape-Sketch-Step: An AI/ML-Based Metaheuristic for Surrogate Optimization Problems

Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization

Hybrid optimal control with mixed-integer Lagrangian methods

A Tutorial on the Design, Experimentation and Application of Metaheuristic Algorithms to Real-World Optimization Problems

An Adaptive Sampling Augmented Lagrangian Method for Stochastic Optimization with Deterministic Constraints

Finding Adversarial Inputs for Heuristics using Multi-level Optimization

New Lagrangian dual algorithms for solving the continuous nonlinear resource allocation problem

Generalizable Meta-Heuristic based on Temporal Estimation of Rewards for Large Scale Blackbox Optimization

Lagrangian Reformulation for Nonconvex Optimization: Tailoring Problems to Specialized Solvers

Enhancing sharp augmented Lagrangian methods with smoothing techniques for nonlinear programming

A New Penalty Dual-Primal Augmented Lagrangian Method and Its Extensions

Hyper-heuristics with low level parameter adaptation

Adaptive Augmented Lagrangian Methods: Algorithms and Practical Numerical Experience

Nature inspired meta heuristic algorithms for optimization problems

Literature Research Optimizer: A New Human-Based Metaheuristic Algorithm for Optimization Problems