Deep Reinforcement Learning Optimized Double Exponentially Weighted Moving Average Controller for Chemical Mechanical Polishing Processes

Zhu Ma,Tianhong Pan,Jiaqiang Tian
DOI: https://doi.org/10.1016/j.cherd.2023.07.049
IF: 4.119
2023-01-01
Chemical Engineering Research and Design
Abstract:This study investigates a deep reinforcement learning (DRL)-assisted double exponentially weighted moving average (dEWMA) controller for run-to-run (RtR) control in the semiconductor manufacturing process. We focus on implementing parameter adaption of dEWMA controllers to achieve disturbance compensation and target tracking. Owing to the powerful adaptive decision-making capability of the DRL, the weight adjustment of dEWMA controller is formulated as a Markov decision process. Specifically, the DRL behaves as an assisted controller to derive appropriate weights that facilitate dEWMA to perform highly accurate disturbance estimation, whereas the standard dEWMA works as a baseline controller to provide suitable recipes for the manufacturing process. Consequently, a composite control strategy integrating DRL and dEWMA is developed. In addition, a twin-delayed deep deterministic policy gradient algorithm is employed to adjust the weights of dEWMA online. The effectiveness of the proposed scheme is validated in a chemical mechanical polishing process. Several disturbance rejection scenarios verify the benefits of the suggested approach.
What problem does this paper attempt to address?