Abstract:Deep Reinforcement Learning (DRL) techniques have received significant attention in control and decision-making algorithms. Most applications involve complex decision-making systems, justified by the algorithms' computational power and cost. While model-based versions are emerging, model-free DRL approaches are intriguing for their independence from models, yet they remain relatively less explored in terms of performance, particularly in applied control. This study conducts a thorough performance analysis comparing the data-driven DRL paradigm with a classical state feedback controller, both designed based on the same cost (reward) function of the linear quadratic regulator (LQR) problem. Twelve additional performance criteria are introduced to assess the controllers' performance, independent of the LQR problem for which they are designed. Two Deep Deterministic Policy Gradient (DDPG)-based controllers are developed, leveraging DDPG's widespread reputation. These controllers are aimed at addressing a challenging setpoint tracking problem in a Non-Minimum Phase (NMP) system. The performance and robustness of the controllers are assessed in the presence of operational challenges, including disturbance, noise, initial conditions, and model uncertainties. The findings suggest that the DDPG controller demonstrates promising behavior under rigorous test conditions. Nevertheless, further improvements are necessary for the DDPG controller to outperform classical methods in all criteria. While DRL algorithms may excel in complex environments owing to the flexibility in the reward function definition, this paper offers practical insights and a comparison framework specifically designed to evaluate these algorithms within the context of control engineering.

A Switching Strategy for Run-to-Run Control Using Deep Deterministic Policy Gradient Algorithm and Integral Controller

Model-Based Ddpg for Motor Control

A Manipulator Control Method Based on Deep Deterministic Policy Gradient with Parameter Noise

A QUOTA-DDPG Controller for Run-to-Run Control

Path Tracking Control of Autonomous Ground Vehicles Via Model Predictive Control and Deep Deterministic Policy Gradient Algorithm

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems

DRL-dEWMA: a Composite Framework for Run-to-run Control in the Semiconductor Manufacturing Process

Target tracking strategy using deep deterministic policy gradient

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

Distributional Reinforcement Learning for Run-to-run Control in Semiconductor Manufacturing Processes

Deep reinforcement learning-assisted extended state observer for run-to-run control in the semiconductor manufacturing process

Data-based Optimal Control for Discrete-time Systems Via Deep Deterministic Policy Gradient Adaptive Dynamic Programming

Train Safety Control Based on Deep Deterministic Policy Gradient with Control Barrier Function

Asynchronous Episodic Deep Deterministic Policy Gradient: Toward Continuous Control in Computationally Complex Environments

Deep Reinforcement Learning with Robust Deep Deterministic Policy Gradient

Swap Softmax Twin Delayed Deep Deterministic Policy Gradient

Deterministic Policy Gradient Adaptive Dynamic Programming for Model-Free Optimal Control

A Policy Gradient Algorithm Integrating Long and Short-Term Rewards for Soft Continuum Arm Control

Path-Tracking Control Strategy of Unmanned Vehicle Based on DDPG Algorithm

Deep Deterministic Policy Gradient and Active Disturbance Rejection Controller based coordinated control for gearshift manipulator of driving robot

Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control