Adaptive Control of an Inverted Pendulum by a Reinforcement Learning-based LQR Method

Ugur Yildiran

2024-09-26

Abstract:Inverted pendulums constitute one of the popular systems for benchmarking control algorithms. Several methods have been proposed for the control of this system, the majority of which rely on the availability of a mathematical model. However, deriving a mathematical model using physical parameters or system identification techniques requires manual effort. Moreover, the designed controllers may perform poorly if system parameters change. To mitigate these problems, recently, some studies used Reinforcement Learning (RL) based approaches for the control of inverted pendulum systems. Unfortunately, these methods suffer from slow convergence and local minimum problems. Moreover, they may require hyperparameter tuning which complicates the design process significantly. To alleviate these problems, the present study proposes an LQR-based RL method for adaptive balancing control of an inverted pendulum. As shown by numerical experiments, the algorithm stabilizes the system very fast without requiring a mathematical model or extensive hyperparameter tuning. In addition, it can adapt to parametric changes online.

Systems and Control

What problem does this paper attempt to address?

This paper aims to solve the control problem of the inverted pendulum system, especially how to achieve adaptive balance control of the inverted pendulum without an accurate mathematical model. Traditional methods usually rely on the mathematical model of the system, but the establishment of these models requires a great deal of manual work, and when the system parameters change, the performance of the controller may decline. In addition, although some studies have used reinforcement learning (RL) - based methods to control the inverted pendulum system, these methods have problems such as slow convergence speed, being easily trapped in local minima, and requiring a large amount of hyper - parameter tuning. To overcome these problems, this paper proposes a reinforcement learning method based on the linear quadratic regulator (LQR). This method can quickly stabilize the system without a mathematical model and can adapt to parameter changes online. Verified by numerical experiments, this algorithm can not only quickly stabilize the inverted pendulum system, but also quickly adjust the control strategy when the parameters change, showing good adaptability and robustness.

Adaptive Control of an Inverted Pendulum by a Reinforcement Learning-based LQR Method

Stabilize the planar single inverted pendulum based on LQR.

Adaptive optimal control of under-actuated robotic systems using a self-regulating nonlinear weight-adjustment scheme: Formulation and experimental verification

Control of double linear inverted pendulum

Application of Deep Reinforcement Learning Control of an Inverted Hydraulic Pendulum

LQR Optimal Control and Embedded System Implementation for a Single Inverted Pendulum

RL Adaptive Fuzzy Control of the Inverted Pendulum System

Design Of Reinforce Learning Control Algorithm And Verified In Inverted Pendulum

Design and Simulation of LQR Controller with the Linear Inverted Pendulum

GA Based LQR Optimal Controller Design for Double Inverted Pendulum

A Q-learning approach to the continuous control problem of robot inverted pendulum balancing

Control of the Double Inverted Pendulum Based on Reinforcement Learning

Control of rotary double inverted pendulum system using LQR sliding surface based sliding mode controller

Stability Analysis of LQR-ANFIS Control Schemes on 2-degree-of-freedom Inverted Pendulum Systems

Swing-up and Balance Control of Inverted Pendulum Based on Reinforcement Learning

Optimal LQR Control of Inverted Pendulum Based on Elitist Preserving Genetic Algorithm

Analysis and Design of an Inverted Pendulum Based on the Robust Linear Quadratic Control

Application of LQR in Inverted Pendulum System

A Novel Recurrent Adaptive Backstepping Optimal Control Strategy for a Single Inverted Pendulum System

Technical Report on Reinforcement Learning Control on the Lucas-Nülle Inverted Pendulum

A New Intelligent Method For The Attitude Control Of Inverted Pendulum