Abstract:Classification, recommendation, and ranking problems often involve competing goals with additional constraints (e.g., to satisfy fairness or diversity criteria). Such optimization problems are quite challenging, often involving non-convex functions along with considerations of user preferences in balancing trade-offs. Pareto solutions represent optimal frontiers for jointly optimizing multiple competing objectives. A major obstacle for frequently used linear-scalarization strategies is that the resulting optimization problem might not always converge to a global optimum. Furthermore, such methods only return one solution point per run. A Pareto solution set is a subset of all such global optima over multiple runs for different trade-off choices. Therefore, a Pareto front can only be guaranteed with multiple runs of the linear-scalarization problem, where all runs converge to their respective global optima. Consequently, extracting a Pareto front for practical problems is computationally intractable with substantial computational overheads, limited scalability, and reduced accuracy. We propose a robust, low cost, two-stage, hybrid neural Pareto optimization approach that is accurate and scales (compute space and time) with data dimensions, as well as number of functions and constraints. The first stage (neural network) efficiently extracts a weak Pareto front, using Fritz-John conditions as the discriminator, with no assumptions of convexity on the objectives or constraints. The second stage (efficient Pareto filter) extracts the strong Pareto optimal subset given the weak front from stage 1. Fritz-John conditions provide us with theoretical bounds on approximation error between the true and network extracted weak Pareto front. Numerical experiments demonstrates the accuracy and efficiency on a canonical set of benchmark problems and a fairness optimization task from prior works.

Deep learning the efficient frontier of convex vector optimization problems

Deep Learning the Efficient Frontier of Convex Vector Optimization Problems

Deep learning for efficient frontier calculation in finance

Exploiting Problem Structure in Deep Declarative Networks: Two Case Studies

Approximating Solutions to the Knapsack Problem using the Lagrangian Dual Framework

Near-Optimal Solutions of Constrained Learning Problems

Deep Learning for Explicitly Modeling Optimization Landscapes

Convex Optimization with an Interpolation-based Projection and its Application to Deep Learning

Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints

Deep Loss Convexification for Learning Iterative Models

A Subgradient-Based Neural Network to Constrained Distributed Convex Optimization

The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models

A Hybrid 2-stage Neural Optimization for Pareto Front Extraction

The Efficient Hedging Frontier with Deep Neural Networks

Efficient Optimization Algorithms for Linear Adversarial Training

Learning structured approximations of combinatorial optimization problems

Self-supervised Equality Embedded Deep Lagrange Dual for Approximate Constrained Optimization

Efficient Convex Algorithms for Universal Kernel Learning

Deep learning enhanced mixed integer optimization: Learning to reduce model dimensionality

Review Non-convex Optimization Method for Machine Learning