Abstract:Classification, recommendation, and ranking problems often involve competing goals with additional constraints (e.g., to satisfy fairness or diversity criteria). Such optimization problems are quite challenging, often involving non-convex functions along with considerations of user preferences in balancing trade-offs. Pareto solutions represent optimal frontiers for jointly optimizing multiple competing objectives. A major obstacle for frequently used linear-scalarization strategies is that the resulting optimization problem might not always converge to a global optimum. Furthermore, such methods only return one solution point per run. A Pareto solution set is a subset of all such global optima over multiple runs for different trade-off choices. Therefore, a Pareto front can only be guaranteed with multiple runs of the linear-scalarization problem, where all runs converge to their respective global optima. Consequently, extracting a Pareto front for practical problems is computationally intractable with substantial computational overheads, limited scalability, and reduced accuracy. We propose a robust, low cost, two-stage, hybrid neural Pareto optimization approach that is accurate and scales (compute space and time) with data dimensions, as well as number of functions and constraints. The first stage (neural network) efficiently extracts a weak Pareto front, using Fritz-John conditions as the discriminator, with no assumptions of convexity on the objectives or constraints. The second stage (efficient Pareto filter) extracts the strong Pareto optimal subset given the weak front from stage 1. Fritz-John conditions provide us with theoretical bounds on approximation error between the true and network extracted weak Pareto front. Numerical experiments demonstrates the accuracy and efficiency on a canonical set of benchmark problems and a fairness optimization task from prior works.

Density Function-Based Trust Region Algorithm for Approximating Pareto Front of Black-Box Multiobjective Optimization Problems

A Trust Region Method for Pareto Front Approximation

A trust-region approach for computing Pareto fronts in multiobjective derivative-free optimization

A Derivative-Free Trust-Region Method for Biobjective Optimization

Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models

A Trust Region Proximal Gradient Method for Nonlinear Multi-objective Optimization Problems

Approximating Multiobjective Optimization Problems with Complex Pareto Fronts

Approximating the Set of Pareto-Optimal Solutions in Both the Decision and Objective Spaces by an Estimation of Distribution Algorithm

Local Model-Based Pareto Front Estimation for Multiobjective Optimization

An Estimation of Distribution Algorithm Based on Multi-Probability Model for Multi-Objective Optimization

Averaged Hausdorff Approximations of Pareto Fronts based on Multiobjective Estimation of Distribution Algorithms

A Trust-Region Algorithm for Noisy Equality Constrained Optimization

A probabilistic framework with the gradient-based method for multi-objective land use optimization

A Modified Probability Collectives Optimization Algorithm Based on Trust Region Method and a New Temperature Annealing Schedule

Adaptive trust region scheme for multi-objective optimization problem using Geršgorin circle theorem

Regional Expected Improvement for Efficient Trust Region Selection in High-Dimensional Bayesian Optimization

An Active-Set Projected Trust Region Algorithm for Box Constrained Optimization Problems.

Algorithms for generating Pareto fronts of multi-objective integer and mixed-integer programming problems

Efficient Trust Region Filter Modeling Strategies for Computationally Expensive Black-Box Optimization

Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models

A Hybrid 2-stage Neural Optimization for Pareto Front Extraction