Abstract:The optimization of controller parameters remains an ongoing challenge in the field of control system applications. This study introduces a novel approach involving the creation of custom actor-critic deep reinforcement learning (DRL) based PID controllers . These controllers are designed with the goal of achieving adaptive tuning, precise trajectory tracking, and stability in a ball-and-plate system. To achieve this objective, multiple actor-critic reinforcement learning agents were developed using different learning algorithms: Soft actor critic (SAC), deep deterministic policy gradient (DDPG), and twin delayed deep deterministic policy gradient (TD3). These agents incorporate multilayer-perceptron (MLP) policy learning algorithms in both the actor and critic network architectures , employing non-linear activation functions. This enables them to fine-tune PID control parameters within an infinite search space. Additionally, a custom reward function derived from the system's environment was integrated into the learning process. The performance of the proposed methods was compared against a benchmark method, specifically, an existing deep reinforcement learning controllers reported in the literature. The evaluation of these controllers and other approaches was based on error metrics and time response analysis. Results demonstrate that the proposed controller denoted as SAC-PID(5) excelled in trajectory tracking and outperformed other methods. It exhibited minimal predictive errors and the shortest time responses in the majority of experiments. This highlights the significance of designing a customized SAC agents with appropriate network architecture, which positively impacts the learning process for intelligent tuning of controllers for classical control systems.

Model Reference Output Feedback Control Using Episodic Natural Actor-Critic

Efficient Reinforcement-Learning Control Algorithm Using Experience Reuse

Natural Gradient Based Reinforcement Learning Algorithm Using Active Stimulating

Learning Linear Parameter-Varying Control of Small-Scale Helicopter Using Episodic Natural Actor-Critic Method

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Actor-Critic Model Predictive Control

Reinforcement Learning Output Feedback NN Control Using Deterministic Learning Technique

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Comparing actor-critic deep reinforcement learning controllers for enhanced performance on a ball-and-plate system

Output Feedback Adaptive Optimal Control of Affine Nonlinear systems with a Linear Measurement Model

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Adaptive Dynamic Programming for Data-Based Optimal State Regulation with Experience Replay.

Robust Actor-Critic with Relative Entropy Regulating Actor

Optimized SAC Deep Reinforcement Learning Control for Electro-hydraulic Servo Systems

Novel Model-free Optimal Active Vibration Control Strategy Based on Deep Reinforcement Learning

Two Dimensional (2D) Feedback Control Scheme Based on Deep Reinforcement Learning Algorithm for Nonlinear Non-repetitive Batch Processes

Model-Based Actor-Critic Learning for Optimal Tracking Control of Robots with Input Saturation.

Autotuning PID control using Actor-Critic Deep Reinforcement Learning

Relaxed Actor-Critic with Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems.

Model-Free Cooperative Optimal Output Regulation for Linear Discrete-Time Multi-Agent Systems Using Reinforcement Learning