Abstract:This research paper introduces a model-free optimal controller for discrete-time Markovian jump linear systems (MJLSs), employing principles from the methodology of reinforcement learning (RL). While Q-learning methods have demonstrated efficacy in determining optimal controller gains for deterministic systems, their application to systems with Markovian switching remains unexplored. To address this research gap, we propose a Q-function involving the Markovian mode. Subsequently, a Q-learning algorithm is proposed to learn the unknown kernel matrix using raw input-state information from the system. Notably, the study proves the convergence of the proposed Q-learning optimal controller gains to the model-based optimal controller gains after proving the convergence of a value iteration algorithm as the first step. Addition of excitation noise to input which is required to ensure the leaning performance does not lead to any bias. Unlike the conventional optimal controller, the proposed method does not require any knowledge on system dynamics and eliminates the need for solving coupled algebraic Riccati equations arising in optimal control of MJLSs. Finally, the efficiency of the proposed method is demonstrated through a simulation study.

A Fuzzy-Model-Based Approach to Optimal Control for Nonlinear Markov Jump Singularly Perturbed Systems: A Novel Integral Reinforcement Learning Scheme

Reinforcement Learning-Based $\mathcal{h}_{\infty }$ Control of 2-D Markov Jump Roesser Systems with Optimal Disturbance Attenuation

A nonlinear predictive control algorithm based on fuzzy online modeling and discrete optimization

Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme

Fuzzy $H_{\infty }$ Control of Discrete-Time Nonlinear Markov Jump Systems via a Novel Hybrid Reinforcement $Q$-Learning Method

Fuzzy-Based Adaptive Optimization of Unknown Discrete-Time Nonlinear Markov Jump Systems With Off-Policy Reinforcement Learning

Reinforcement learning‐based composite suboptimal control for Markov jump singularly perturbed systems with unknown dynamics

Optimal Vibration Control of a Class of Nonlinear Stochastic Systems with Markovian Jump

H∞$$ {h}_{\infty } $$ Optimal Output Tracking Control for Markov Jump Systems: A Reinforcement Learning‐based Approach

Decentralized Optimization for Fuzzy Singularly Perturbed Interconnected Systems: A Hybrid Reinforcement Learning Method

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Fuzzy-Model-Based Nonfragile Control for Nonlinear Singularly Perturbed Systems With Semi-Markov Jump Parameters

Nonfragile Output Feedback Tracking Control for Markov Jump Fuzzy Systems Based on Integral Reinforcement Learning Scheme

Nonfragile $\mathcal{h}_{\infty}$ Control for Fuzzy Markovian Jump Systems under Fast Sampling Singular Perturbation

Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach

Finite-Time Control of Markov Jump Lur'e Systems with Singular Perturbations

Fuzzy multi-objective fault-tolerant control for nonlinear Markov jump singularly perturbed systems with persistent dwell-time switched transition probabilities

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

Reinforcement Learning-Based Direct Adaptive Optimal Control of JLQ Model

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Finite-time L2−l∞ Tracking Control for Markov Jump Repeated Scalar Nonlinear Systems with Partly Usable Model Information