Non-myopic Bayesian optimization using model-free reinforcement learning and its application to optimization in electrochemistry

Mujin Cheon,Haeun Byun,Jay H. Lee
DOI: https://doi.org/10.1016/j.compchemeng.2024.108624
IF: 4.13
2024-02-09
Computers & Chemical Engineering
Abstract:Bayesian Optimization (BO) is a robust tool for tackling black-box optimization problems, yet traditional acquisition functions often suffer from a short-sighted approach, leading to suboptimal sampling. In this study, we introduce a novel methodology that defines a Markov Decision Process (MDP) tailored for BO, enabling the incorporation of Reinforcement Learning (RL) into the BO framework. Our RL-based BO method strategically evaluates the utility of current data acquisition in the context of future decision making, this forward-looking perspective ensures that data collected at present is optimally leveraged in subsequent steps, resulting in a more data-efficient and effective sampling strategy compared to traditional BO methods. Our method has been tested across a range of benchmark functions, it consistently demonstrates superior data efficiency over existing BO algorithms, irrespective of the nature of the underlying function. We also explore how adjusting the lookahead horizon influences the performance of our RL-BO approach. Our RL-BO method is further applied to an Ag/C catalyst optimization problem to compare its data efficiency with other BO strategies, further showcasing the method's adaptability to complex real-world problems.
engineering, chemical,computer science, interdisciplinary applications
What problem does this paper attempt to address?