Abstract:This paper proposes a multi-objective reinforcement learning algorithm (MORLA) and uses simultaneous perturbation stochastic approximation (SPSA) to improve the convergence of it. Usually, reinforcement learning (RL) is used to design neurocontroller for control system with single objective. When facing multi-objective system, it is necessary to design the neurocontroller according to the personal preference. The MORLA can transform the multi-objective into synthetical objective and applies parallel genetic algorithm (PGA) to evolve the neurocontroller according to the synthetical objective. To establish the synthetical objective, the objective weight which represents the personal preference is calculated by solving the constrained optimization problem (COP) at the end of each generation. The COP requires not only the biggest variance of the synthetical objective in the population, but also requires the weight to fit the designer's preference. After acquiring the weights, the PGA can select the elitists from the population according to the designer's preference and design a satisfying neurocontroller by evolutionary operations. In addition, although GA has good global search ability, it descends slowly at local area. This paper applies SPSA algorithm to search optimal solution when GA is vibrating at local area. The SPSA converges fast by efficient gradient approximation that relies on measurements of the objective function. The hybrid algorithm accelerates the learning speed of reinforcement learning. At last, the MORLA is used to design neurocontroller for a speed-controlled induction motor drive with indirect vector control. With different personal preferences for the drive system, the simulation results show the feasibility and validity of the MORLA. © 2011 IEEE.

Multi-objective reinforcement learning for fed-batch fermentation process control

Batch-To-Batch Control of Batch Processes Based on Multilayer Recurrent Fuzzy Neural Network

Safe Multiagent Learning with Soft Constrained Policy Optimization in Real Robot Control

Multi-objective reinforcement learning algorithm and its improved convergency method

Ai-based optimal control of fed-batch biopharmaceutical process leveraging deep reinforcement learning

Reinforcement learning for batch bioprocess optimization

Multi-objective Reinforcement Learning Algorithm and Its Application in Drive System

Prediction Guided Meta-Learning for Multi-Objective Reinforcement Learning

Meta Proximal Policy Optimization for Cooperative Multi-Agent Continuous Control

C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Front

Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning

Learning Pareto Set for Multi-Objective Continuous Robot Control

Multistage Optimal Control for Microbial Fed-Batch Fermentation Process

A Two-Stage Multi-Objective Deep Reinforcement Learning Framework.

Optimal Control of Fed-Batch Processes Based on Multiple Neural Networks

Multi-objective Optimisation of Nonlinear Switched Systems in Uncoupled Fed-Batch Fermentation.

Batch to Batch Iterative Learning Control of a Fed-Batch Fermentation Process Using Linearised Models

Policy manifold generation for multi-task multi-objective optimization of energy flexible machining systems

Multiobjective optimization and nonlinear model predictive control of the continuous fermentation process involving Saccharomyces Cerevisiae

Batch to Batch Iterative Learning Control of a Fed-Batch Fermentation Process

Multi-objective Optimization of a Nonlinear Switched Time-Delay System in Microbial Fed-Batch Process