Abstract:Prescriptive analytics provides organizations with scalable solutions for large-scale, automated decision making. At the core of prescriptive analytics methodology is optimization, a field devoted to the study of algorithms that solve complex decision-making problems. Optimization algorithms rely heavily on generic methods for identifying tight bounds, which provide both solutions to problems and optimality guarantees. In the last decade, decision diagrams (DDs) have demonstrated significant advantages in obtaining bounds compared with the standard linear relaxation commonly used by commercial solvers. However, the quality of the bounds computed by DDs depends heavily on the variable ordering chosen for the construction. Besides, the problem of finding an ordering that optimizes a given metric is generally NP-hard. This paper studies how machine learning, specifically deep reinforcement learning (DRL), can be used to improve bounds provided by DDs, in particular through learning a good variable ordering. The introduced DRL models improve primal and dual bounds, even over standard linear programming relaxations, and are integrated in a full-fledged branch-and-bound algorithm. This paper, therefore, provides a novel mechanism for utilizing machine learning to tighten bounds, adding to recent research on using machine learning to obtain high-quality heuristic solutions and, for the first time, using machine learning to improve relaxation bounds through a generic bounding method. We apply the methods on a classic optimization problem, the maximum independent set, and demonstrate through computational testing that optimization bounds can be significantly improved through DRL. We provide the code to replicate the results obtained on the maximum independent set. Summary of Contribution: This paper studies the use of reinforcement learning to compute a variable ordering of decision diagram-based approximations for discrete optimization problems. This is among the first works to propose the use of machine learning to improve upon generic bounding methods for discrete optimization problems, thereby establishing a critical bridge between optimization and learning.

A Novel Framework Using Deep Reinforcement Learning for Join Order Selection

Learning to Optimize Join Queries With Deep Reinforcement Learning

Reinforcement Learning with Tree-LSTM for Join Order Selection

Deep Reinforcement Learning for Join Order Enumeration

Query Join Order Optimization Method Based on Dynamic Double Deep Q-Network

Join Query Optimization with Deep Reinforcement Learning Algorithms

JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning

Session-based Interactive Recommendation Via Deep Reinforcement Learning

A Novel Reinforcement Learning Approach for Enhancing Flexible Job-Shop Scheduling with Dual Gated-Attention Network

A New Graph Neural Network-based Join Optimization Algorithm

Solving non-permutation flow-shop scheduling problem via a novel deep reinforcement learning approach

RLO: a Reinforcement Learning-Based Method for Join Optimization

Team Formation in Large Organizations: A Deep Reinforcement Learning Approach

A Novel Multi-Step Q-learning Method to Improve Data Efficiency for Deep Reinforcement Learning.

Graph Ordering: Towards the Optimal by Learning

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

ACDRL: A Constraint-guided Dynamic Reinforcement Learning for Index Selection

A Novel Tree-Based Method for Interpretable Reinforcement Learning

A novel reinforcement learning-based method for structure optimization

Exploring Dynamic Selection of Branch Expansion Orders for Code Generation

Improving Variable Orderings of Approximate Decision Diagrams Using Reinforcement Learning