Abstract:This research paper introduces a model-free optimal controller for discrete-time Markovian jump linear systems (MJLSs), employing principles from the methodology of reinforcement learning (RL). While Q-learning methods have demonstrated efficacy in determining optimal controller gains for deterministic systems, their application to systems with Markovian switching remains unexplored. To address this research gap, we propose a Q-function involving the Markovian mode. Subsequently, a Q-learning algorithm is proposed to learn the unknown kernel matrix using raw input-state information from the system. Notably, the study proves the convergence of the proposed Q-learning optimal controller gains to the model-based optimal controller gains after proving the convergence of a value iteration algorithm as the first step. Addition of excitation noise to input which is required to ensure the leaning performance does not lead to any bias. Unlike the conventional optimal controller, the proposed method does not require any knowledge on system dynamics and eliminates the need for solving coupled algebraic Riccati equations arising in optimal control of MJLSs. Finally, the efficiency of the proposed method is demonstrated through a simulation study.

Reinforcement Learning-Based Direct Adaptive Optimal Control of JLQ Model

Reinforcement Learning-Based $\mathcal{h}_{\infty }$ Control of 2-D Markov Jump Roesser Systems with Optimal Disturbance Attenuation

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach

Optimization For Controlled Jump Rates Of Jlqg Problem

Reinforcement Learning with Adjustable Convergence Rate for Data-Based Nonlinear Control

Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme

Fuzzy-Based Adaptive Optimization of Unknown Discrete-Time Nonlinear Markov Jump Systems With Off-Policy Reinforcement Learning

Reinforcement Learning-Based Optimal Control for Markov Jump Systems with Completely Unknown Dynamics

Optimization of dependently controlled jump rates of JLQG problem

Learning Algorithm for LQG Model with Constrained Control

Online adaptive optimal control for continuous-time Markov jump linear systems using a novel policy iteration algorithm

Policy Gradient Reinforcement Learning for Parameterized Continuous-Time Optimal Control

A Fuzzy-Model-Based Approach to Optimal Control for Nonlinear Markov Jump Singularly Perturbed Systems: A Novel Integral Reinforcement Learning Scheme

A Reinforcement Learning Method for LQR Control Problem

Reinforcement Learning and Adaptive Optimization of a Class of Markov Jump Systems with Completely Unknown Dynamic Information

Reinforcement learning‐based composite suboptimal control for Markov jump singularly perturbed systems with unknown dynamics

Fuzzy H∞ Control of Discrete-Time Nonlinear Markov Jump Systems via a Novel Hybrid Reinforcement Q-Learning Method

Linear Quadratic Tracking Control of Unknown Systems: A Two-Phase Reinforcement Learning Method.

Hybrid Reinforcement Learning for Optimal Control of Non-Linear Switching System

Learning Optimal Control Policy for Unknown Discrete-Time Systems